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What  is  The  Nation's  Report  Card? 

THE  NATION’S  REPORT  CARD,  the  National  Assessment  of  Educational  Progress  (NAEP),  is  the  only  nationally 
representative  and  continuing  assessment  of  what  America’s  students  know  and  can  do  in  various  subject  areas.  Since  1969, 
assessments  have  been  conducted  periodically  in  reading,  mathematics,  science,  writing,  history,  geography,  and  other 
fields.  By  making  objective  information  on  student  performance  available  to  policymakers  at  the  national,  state,  and  local 
levels,  NAEP  is  an  integral  part  of  our  nation’s  evaluation  of  the  condition  and  progress  of  education.  Only  information 
related  to  academic  achievement  is  collected  under  this  program.  NAEP  guarantees  the  privacy  of  individual  students  and 
their  families. 

NAEP  is  a congressionally  mandated  project  of  the  National  Center  for  Education  Statistics,  the  U.S.  Department  of 
Education.  The  Commissioner  of  Education  Statistics  is  responsible,  by  law,  for  carrying  out  the  NAEP  project  through 
competitive  awards  to  qualified  (^ganizations.  NAEP  reports  directly  to  the  Commissioner,  who  is  also  responsible 
for  providing  continuing  reviews,  including  validation  studies  and  solicitation  of  public  comment,  on  NAEP’s  conduct 
and  usefulness. 

In  1988,  Congress  established  the  National  Assessment  Governing  Board  (NAGB)  to  formulate  policy  guidelines  for 
NAEP.  The  Board  is  responsible  for  selecting  the  subject  areas  to  be  assessed  from  among  those  included  in  the  National 
Education  Goals;  for  setting  appropriate  student  performance  levels;  for  developing  assessment  objectives  and  test 
specifications  through  a national  consensus  approach;  for  designing  the  assessment  methodology;  for  developing  guidelines 
for  reporting  and  disseminating  NAEP  results;  for  developing  standards  and  procedures  for  interstate,  regional,  and  national 
comparisons;  for  determining  the  appropriateness  of  test  items  and  ensuring  they  are  free  from  bias;  and  for  taking  actions 
to  improve  the  form  and  use  of  the  National  Assessment. 
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THE  NAEP  1996  TECHNICAL  REPORT 

Introduction^ 

James  E.  Carlson 
Educational  Testing  Service 

The  1996  Nation2il  Assessment  of  Educational  Progress  (NAEP)  monitored  the  performance  of 
students  in  American  schools  in  the  subject  areas  of  reading,  mathematics,  science,  and  writing.  The 
national  sample  involved  nearly  124,000  public  and  nonpublic-school  students  who  were  9-,  13-,  or  17 
years  old  or  in  grades  4,  8,  or  12. 

The  purpose  of  this  technical  report  is  to  provide  details  on  the  instrument  development,  sample 
design,  data  collection,  and  data  analysis  procedures  of  the  1996  assessment.  Detailed  substantive  results 
are  not  presented  here  but  can  be  found  in  a series  of  NAEP  reports  on  the  status  of  and  trends  in  student 
performance;  several  additional  reports  provide  information  on  how  the  assessment  was  designed  and 
implemented.  The  reader  is  directed  to  the  following  reports  for  1996  results  and  supporting 
documentation: 

• NAEP  1996  Mathematics  Report  Card  for  the  Nation  and  the  States:  Findings  from 
the  National  Assessment  of  Educational  Progress  (Reese,  Miller,  Mazzeo,  & Dossey, 

1997) 

• NAEP  1996  Science  Report  Card  for  the  Nation  and  the  States:  Findings  from  the 
National  Assessment  of  Educational  Progress  (O’Sullivan,  Reese,  & Mazzeo,  1997) 

• The  NAEP  Guide:  A Description  of  the  Content  and  Methods  of  the  1994  and  1996 

Assessments  1996) 

• NAEP  1996  Trends  in  Academic  Progress:  Achievement  of  US,  Students  in  Science, 

1969  to  1996;  Mathematics,  1973  to  1996;  Reading,  1971  to  1996;  and  Writing, 

1984  to  1996  (Campbell,  Voelkl,  & Donahue,  1997) 

• NAEP  1996  Mathematics  Cross-State  Data  Compendium  for  the  Grade  4 and  Grade 
8 Assessment  (Shaughnessy,  Nelson,  & Norris,  1997) 

• NAEP  1996  Science  Cross-State  Data  Compendium  for  the  Grade  8 Assessment 
(Keiser,  Nelson,  Norris,  & Szyszkiewicz,  1998) 

• Mathematics  Framework  for  the  1996  National  Assessment  of  Educational  Progress 
(National  Assessment  Governing  Board,  1994) 

• Science  Framework  for  the  1996  National  Assessment  of  Educational  Progress 
(National  Assessment  Governing  Board,  1993) 
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• Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Mathematics 
(Allen,  Jenkins,  Kulick,  & Zelenak,  1997) 

• Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Science  (Allen, 
Swinton,  Isham,  & Zelenak,  1998) 

• NAEP  1996  National  Assessment  Secondary-Use  Data  Files  User  Guide  (Rogers, 
Kline,  & Schoeps,  1999) 

• NAEP  1996  State  Assessment  Program  in  Mathematics  Secondary-Use  Data  Files 
User  Guide  (OReilly,  Zelenak,  Rogers,  & Kline,  1999) 

• NAEP  1996  State  Assessment  Program  in  Science  Secondary-Use  Data  Files  User 
Guide  (OReilly,  Zelenak,  Rogers,  & Kline,  1999) 

• NAEP  1996  Science  Performance  Standards:  Achievement  Results  for  the  Nation 
and  the  States  (Bourque,  Champagne,  & Crissman,  1997) 

• School  Policies  Affecting  Instruction  in  Mathematics:  Findings  from  the  National 
Assessment  of  Educational  Progress  (Hawkins,  Stancavage,  & Dossey,  1998) 

• Student  Work  and  Teacher  Practices  in  Mathematics  (Mitchell,  Hawkins,  Jakwerth, 
Stancavage,  & Dossey,  1999) 

• Estimation  Skills,  Mathematics-in-context,  and  Advanced  Skills  in  Mathematics 
(Hawkins,  Mitchell,  Stancavage,  & Dossey,  1999) 

• Students  Learning  Science:  A Report  on  Policies  and  Practices  in  U.S.  Schools 
(O’Sullivan,  Weiss,  & Askew,  1998) 

• Student  Work  and  Teacher  Practices  in  Science:  A Report  on  What  Students  Know 
and  Can  Do  (O’Sullivan  & Weiss,  1999) 

• The  1996  NAEP  Sampling  and  Weighting  Report  (Wallace  & Rust,  1999) 

• Report  on  Data  Collection  Activities  for  the  1996  National  Assessment  of 
Educational  Progress  (Westat,  Inc.,  1996) 

• Report  of  Processing  and  Professional  Activities  (National  Computer  Systems,  1996) 


The  Report  Card  publications  highlight  results  for  the  nation,  states,  and  selected  subgroups. 
Reports  on  student  work  and  teacher  practices  focus  on  instructional  variables  related  to  mathematics  and 
science  education  and  are  designed  to  meet  the  information  needs  of  teachers  and  curriculum  specialists. 
The  aim  of  the  reports  on  school  policies,  which  focus  on  instruction-relevant  variables  from  the  school 
or  community  level,  is  to  meet  the  information  needs  of  principals,  school  boards,  and  interested  citizens. 
Technical  and  other  reports  listed  above  provide  more  detailed  information  on  the  NAEP  data  and 
analysis  procedures.  Many  of  the  NAEP  reports,  including  the  almanacs  (summary  data  tables),  are  also 
available  on  the  Internet  at  http://nces.ed.gov/naep.  For  ordering  information  on  printed  copies  of  these 
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reports,  go  to  the  Department  of  Education  web  page  http://www.ed.gov/pubs/edpubs.html,  call  toll  free 
1_877-  4ED  PUBS  (877^33-7827),  or  write  to: 

Education  Publications  Center  (ED  Pubs) 

U.S.  Department  of  Education 

P.O.Box  1398 

Jessup,  MD  20794-1398 

The  Frameworks  are  designed  to  assess  the  outcomes  of  students’  education  in  mathematics 
and  science  in  grade  4,  8,  and  12  as  part  of  NAEP.  For  ordering  information  on  these  reports,  write: 

National  Assessment  Governing  Board 
800  North  Capitol  Street  NW 
Suite  825 

Washington,  DC  20002 

The  Frameworks  and  other  NAGB  documents  are  also  available  through  the  web  at 
http://www.nagb.org. 

Additional  samples  of  approximately  125,000  fourth-  and  125,000  eighth-graders  in  48 
jurisdictions  were  assessed  in  the  1996  state  assessment  in  mathematics.  Also  a sample  of 
approximately  125,000  fourth-graders  in  47  states  and  jurisdictions  was  assessed  as  part  of  the  1996 
state  assessment  in  science.  A representative  sample  of  about  2,500  students  was  selected  in  each 
jurisdiction  for  each  subject  at  each  grade  level.  The  state-level  sampling  plan  allowed  for  cross-state 
comparisons  and  comparisons  with  the  nation  in  fourth-grade  science  and  fourth-  and  eighth-grade 
mathematics  achievement.  Technical  details  of  the  state  assessments  are  not  presented  in  this  technical 
report  but  can  be  found  in  the  state  technical  report^^. 


AN  OVERVIEW  OF  NAEP  IN  1996 


For  the  1996  assessment,  NAEP  researchers  continued  to  build  on  the  original  design 
technology  outlined  in  A New  Design  for  a New  Era  (Messick,  Beaton,  & Lord,  1983).  In  order  to 
maintain  its  links  to  the  past  and  still  implement  innovations  in  measurement  technology,  NAEP 
continued  its  multistage  sampling  approach.  Long-term  trend  and  short-term  trend  samples  use  the 
same  methodology  and  population  definitions  as  in  previous  assessments.  Main  assessment  samples 
use  innovations  associated  with  new  NAEP  technology  and  address  current  educational  issues.  Long- 
term trend  data  are  used  to  estimate  changes  in  performance  from  previous  assessments;  main 
assessment  sample  data  are  used  primarily  for  analyses  involving  the  current  student  population,  but 
also  to  estimate  short-term  trends  for  a small  number  of  recent  assessments.  In  continuing  to  use  this 
two-tiered  approach,  NAEP  reaffirms  its  commitment  to  maintaining  long-term  trends  while  at  the 
same  time  implementing  the  latest  in  measurement  technology. 


A major  new  design  feature  was  introduced  for  1996  to  permit  the  introduction  of  new  inclusion 
rules  for  students  with  disabilities  (SD)  and  limited  English  proficient  (LEP)  students,  and  the 
introduction  of  testing  accommodations  for  those  students.  The  1996  national  NAEP  incorporated  a 
multiple  sampling  plan  that  allowed  for  the  study  of  changes  in  NAEP  inclusion  and  accommodation 
procedures.  In  order  to  provide  for  studies  of  the  effects  of  these  changes,  students  from  different 
samples  were  administered  the  NAEP  instruments  using  different  sets  of  inclusion  rules  and 
accommodation  procedures.  Testing  accommodations  were  provided  for  SD  and  LEP  students  in  certain 
samples  who  could  be  assessed,  but  not  with  standard  instruments  or  administration  procedures. 
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In  the  1996  assessment,  many  of  the  innovations  that  were  implemented  for  the  first  time  in  1988 
were  continued  and  enhanced.  For  example,  a variant  of  the  focused  balanced  incomplete  block  (focused- 
BIB)  booklet  design  that  was  used  in  1988  and  has  continued  to  be  used  in  other  assessment  years,  was 
used  in  the  1996  main  assessment  samples  in  mathematics  and  science.  In  the  focused-BIB  design,  an 
individual  receives  blocks  of  cognitive  items  in  the  same  subject  area.  The  focused-BIB  design  allows  for 
improved  estimation  within  a particular  subject  area,  and  estimation  continues  to  be  optimized  for  groups 
rather  than  individuals. 

In  1996,  NAF.P  continued  to  apply  the  plausible  values  approach  to  estimating  means  for 
demographic  as  well  as  curriculum-related  subgroups.  Proficiency  estimates  were  based  on  draws  from  a 
posterior  distribution  that  was  based  on  an  optimum  weighting  of  two  sets  of  information;  the  student’s 
responses  to  cognitive  items,  and  his  or  her  demographic  and  associated  educational  process  variables. 
This  Bayesian  procedure  was  developed  by  Mislevy  (see  Chapter  1 1 or  Mislevy,  1991).  The  1996 
procedures  continued  to  use  an  improvement  that  was  implemented  first  in  1988  and  refined  for  the  1994 
assessment.  This  is  a multivtu-iate  procedure  that  uses  information  from  all  scales  within  a given  subject 
area  in  the  estimation  of  the  proficiency  distribution  on  any  one  scale  in  that  subject  area. 

A major  improvement  used  in  the  1992  and  1994  assessments,  and  continued  in  1996,  was  the 
use  of  the  generalized  partial  credit  model  for  item  response  theory  (IRT)  scaling.  This  allowed  the 
incorporation  of  constructed-response  questions  that  are  scored  on  a multipoint  rating  scale  into  the 
NAEP  scale  in  a way  that  utilizes  the  information  available  in  each  response  category. 

One  important  innovation  in  reporting  the  1990  assessment  data  that  was  continued  through  1996 
was  the  use  of  simultaneous  comparison  procedures  in  carrying  out  significance  tests  for  the  differences 
across  assessment  years.  Methods  such  as  the  Bonferroni  allow  one  to  control  for  the  type  I error  rate  for 
a fixed  number  of  comparisons.  In  1996,  a new  procedure  that  provided  more  powerful  procedures  that, 
control  for  the  false  discovery  rate  were  implemented  for  some  comparisons.  Tests  for  linear  and 
quadratic  trends  were  also  applied  to  the  national  trend  data  in  reading,  mathematics,  science,  and 
writing. 


ORGANIZATION  OF  THE  TECHNICAL  REPORT 

Part  I of  this  report  presents  the  details  of  the  design  of  the  1996  National  Assessment, 
sutnmtu'ized  in  Chapter  1.  Chapters  2 through  8 describe  the  development  of  the  objectives  and  the  items 
used  in  the  assessment,  the  sample  selection  procedures,  the  assessment  booklets  and  questionnaires,  the 
administration  of  the  assessment  in  the  field,  the  processing  of  the  data  from  the  assessment  instruments 
into  computer-readable  form,  the  professional  scoring  of  constructed-response  items,  and  the  methods 
used  to  create  a complete  NAEP  database. 

The  1996  NAEP  data  analysis  procedures  are  described  in  Part  II  of  the  report.  Chapter  9 
provides  a summary  of  the  analysis  steps.  Subsequent  chapters  provide  a general  discussion  of  the 
weighting  and  variance  estimation  procedures  used  in  NAEP,  an  overview  of  NAEP  scaling 
methodology,  and  details  of  the  trend  and  main  assessment  analyses  performed  for  each  subject  area  in 
the  1996  assessment. 

Chapter  19  presents  basic  data  from  the  1996  assessment,  including  the  properties  of  the 
measuring  instruments  and  characteristics  of  the  sample. 
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Chapter  1 


OVERVIEW  OF  PART  I:  THE  DESIGN  AND  IMPLEMENTATION 

OF  THE  1996  NAEP^ 

Nancy  L.  Allen  and  Eugene  G.  Johnson 
Educational  Testing  Service 


1.1  INTRODUCTION 

The  1996  National  Assessment  collected  information  on  the  knowledge,  skills,  and  attitudes  of 
young  Americans  in  mathematics,  science,  reading,  and  writing.  The  three  components  of  the  National 
Assessment  were  the  main  assessments  of  mathematics  and  science,  the  long-term  trend  assessments  of 
mathematics,  science,  reading,  and  writing,  and  special  assessments  of  aspects  of  mathematics  and 
science.  The  basis  for  the  information  collected  for  the  National  Assessment  was  a complex  sample 
survey  involving  nearly  124,000  students,  consisting  of  national  samples  of  public-  and  nonpublic-school 
students  who  were  in  grades  4,  8,  and  12  or  were  9-,  13-,  or  17-year  olds.  Additional  NAEP  data  came 
from  the  State  Assessment  program,  which  in  1996  assessed  mathematics  at  grades  4 and  8 in 
representative  samples  of  public-  and  nonpublic-school  students  in  44  states,  the  District  of  Columbia, 
Department  of  Defense  Domestic  Dependent  Elementary  and  Secondary  Schools  (DDESS),  Department 
of  Defense  Dependents  Schools  (DoDDS),  and  Guam.  The  1996  State  Assessment  program  also  assessed 
science  at  grade  8 in  representative  samples  of  public-  and  nonpublic-school  students  in  43  states,  the 
District  of  Columbia,  DDESS,  DoDDS,  and  Guam;  DDESS  and  DoDDS  fourth-grade  students  were 
assessed  as  part  of  a separate  special  science  assessment. 

This  chapter  describes  the  design  for  the  1996  assessment  and  gives  an  overview  of  the  steps 
involved  in  its  implementation,  from  the  planning  stage  through  the  creation  of  edited  data  files.  The 
major  components  of  the  implementation  are  presented  here  with  references  to  other  chapters  in  Part  I 
that  provide  greater  detail  on  each  aspect  of  the  assessment.  The  procedures  used  for  the  analysis  of  the 
data  are  summarized  in  the  overview  to  Part  II  and  discussed  in  detail  in  the  remaining  chapters  in  that 
part  of  the  report.  Excluded  from  this  technical  report  are  the  details  of  the  design  and  analysis  of  the 
1996  State  Assessments,  which  instead  appear  in  the  Technical  Report  of  the  NAEP  1996  State 
Assessment  Program  in  Mathematics  (Allen,  Jenkins,  Kulick,  & Zelenak,  1997),  and  in  the  Technical 
Report  of  the  NAEP  1996  State  Assessment  Program  in  Science  (Allen,  Swinton,  Isham,  & Zelenak, 
1998).  Also  excluded  are  the  details  of  the  analyses  of  special  studies  of  advanced  matheniatics  and 
science  students  and  of  students  receiving  special  theme-related  and  estimation  mathematics  items.  The 
analyses  will  be  described  in  the  appendices  of  the  reports  containing  the  results  of  these  special  studies. 


' Nancy  L.  Allen  is  responsible  for  the  psychometric  and  statistical  analysis  of  national  and  state  NAEP  data.  Eugene  G.  Johnson 
is  a senior  psychometrician,  contributing  to  the  design  of  NAEP  and  to  discussions  of  sampling  issues.  Previously,  he  was 
responsible  for  the  psychometric  and  statistical  analysis  of  NAEP  data.  The  authors  are  indebted  to  the  authors  of  Chapters  2 
through  8 for  portions  of  this  chapter. 
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The  organization  of  this  chapter,  and  of  Part  I,  is  as  follows: 

• Section  1.2  provides  an  overview  of  the  NAEP  design  for  1996  and  includes  a 
description  of  the  constituent  samples.  To  provide  background  information,  the 
section  also  includes  the  assessment  schedule  from  the  inception  of  NAEP  in  1969 
through  the  1996  assessment. 

• Section  1.3  provides  a summary  of  the  development  of  the  objectives  for  each 
subject  area  in  the  assessment  and  a description  of  the  development  and  review  of 
the  items  written  to  fit  those  objectives.  Details  of  the  objective  and  item 
development  processes  appear  in  Chapter  2. 

• Section  1.4  provides  a summary  of  the  sampling  design  used  for  the  1996  assessment 
with  a fuller  description  provided  in  Chapter  3. 

• Section  1.5  includes  a discussion  of  the  assignment  of  the  cognitive  and  background 
questions  to  assessment  booklets  and  a description  of  the  complex  block  designs  that 
were  the  basis  for  assigning  cognitive  items  to  assessment  booklets  and  assessment 
booklets  to  individuals.  Chapter  4 provides  a detailed  description  of  the  assessment 
booklets. 

• Section  1.6  provides  a summary  of  the  field  administration  procedures,  including  the 
processes  of  training  field  administrators,  attaining  school  cooperation, 
administering  the  assessment,  and  conducting  quality  control.  Further  details  appear 
in  Chapter  5. 

• Section  1.7  includes  a description  of  the  flow  of  data  from  the  receipt  of  the 
assessment  materials  through  data  entry,  validation,  and  resolution  to  the  creation  of 
edited  data  files.  Chapter  6 provides  a detailed  description  of  the  process. 

• Section  1.8  contains  a discussion  of  the  professional  scoring  of  students’  responses 
to  the  constructed-response  items  in  the  assessment.  Details  of  the  process  are  given 
in  Chapter  7. 

• Section  1 .9  provides  a summary  of  the  creation  of  the  database,  the  quality  control  of 
data  entry,  and  lists  the  1996  database  products.  This  section  also  includes  a 
description  of  the  use  of  the  World  Wide  Web  for  dissemination  of  NAEP 
information.  Further  details  appear  in  Chapter  8. 


1.2  THE  1996  NAEP  DESIGN 

A major  purpose  of  NAEP  is  the  reliable  measurement  of  trends  in  educational  achievement  over 
time.  To  do  this  well,  confounding  effects  due  to  changes  from  one  assessment  to  the  next  in  assessment 
instrumentation  or  in  assessment  procedures  must  be  minimized.  This  implies  a stability  in  the 
measurement  process  over  time.  At  the  same  time,  the  assessment  must  remain  current  by  allowing  the 
introduction  of  new  curriculum  concepts  and  changes  in  educational  priorities  and  by  permitting  the  use 
of  new  measurement  technology.  The  objectives  for  an  assessment  are  determined  through  a consensus 
process  in  which  committees  of  subject  matter  experts,  scholars,  and  citizens  representing  many  diverse 
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constituencies  and  points  of  view  are  assembled  to  determine  the  educational  goals  that  students  should 
achieve.  Satisfying  these  objectives  often  requires  changes  in  assessment  instrumentation  and 
methodology. 

In  order  to  meet  the  goals  of  measuring  trends  reliably  and  responding  to  changes  in  the  current 
thinking  about  subject  areas,  NAEP  has  instituted  a multicomponent  assessment  system  where  each 
component  is  itself  a set  of  assessments  designed  to  accomplish  a specific  goal.  There  are  three 
components  in  the  1996  National  Assessment  design:  (1)  main  assessments;  (2)  assessments  for  long- 
term trend;  and  (3)  special  assessments.  In  particular,  the  main  assessments  respond  to  changes  in 
curriculum  on  a regular  basis,  while  the  long-term  trend  assessments  are  not  changed  and  measure 
longer-term  trends  in  a valid  way.  These  are  discussed  in  detail  in  this  chapter. 

Several  improvements  were  made  in  the  design  of  NAEP  in  the  1984  and  succeeding 
assessments.  Until  the  1984  assessment,  NAEP  was  administered  using  matrix  sampling  and  tape 
recorders;  that  is,  by  administering  booklets  of  exercises  using  an  aurally  presented  stimulus  that  paced 
groups  of  students  through  the  individual  assessment  exercises  in  a common  booklet.  In  the  1984 
assessment,  BIB  spiraling,  which  does  not  include  aural  pacing,  was  introduced  in  place  of  taped  matrix 
sampling.  The  NAEP  design  now  includes  sampling  grade  populations  as  well  as  the  age  populations  that 
NAEP  originally  assessed.  The  definitions  of  student  age  and  the  time  of  year  in  which  the  assessment 
takes  place  have  been  made  uniform  so  that  students  in  the  fourth-,  eighth-,  and  twelfth-grades  are 
assessed.  To  shorten  the  timetable  for  reporting  results,  the  period  for  national  data  collection  was 
decreased  in  1992,  1994,  and  1996  from  the  five-month  period  used  in  1990  to  a three-month  period  in 
the  winter  (corresponding  to  the  period  used  for  the  winter  half-sample  of  the  1990  National 
Assessment).  To  enhance  the  coverage  of  the  subject  areas  assessed,  the  number  of  items  measuring 
knowledge  and  skills  was  increased  for  the  1992,  1994,  and  1996  assessments. 

A special  feature  of  the  1996  main  and  state  assessments  was  the  introduction  of  new  rules  for 
inclusion  of  students  with  disabilities  (SD)  and  limited  English  proficient  (LEP)  students  in  NAEP 
assessments  (presented  in  Chapter  5).  A subsample  of  the  schools  selected  for  participation  in  the  1996 
assessments  used  the  old  inclusion  rules  (sample  type  1;  SI)  to  determine  whether  students  should  be 
included  in  the  assessment  and  another  subsample  used  the  new  inclusion  rules  (sample  type  2;  S2).  In 
addition  to  the  two  groups  of  schools  using  the  old  and  new  inclusion  rules  without  offering  students 
special  testing  accommodations,  the  1996  main  assessments  included  a third  group  of  schools  that  used 
the  new  inclusion  rules  and  offered  students  within  those  schools  accommodations  to  the  standard  NAEP 
administration  procedures  (sample  type  3;  S3).  Figure  1-1  contains  the  layout  of  the  pieces  of  the  sample 
collected  for  each  grade  of  the  main  assessment  of  mathematics. 

The  accommodations  provided  by  NAEP  in  the  main  assessments  were  meant  to  match  those 
specified  in  the  student’s  individualized  education  program  (lEP)  or  those  ordinarily  provided  in  the 
classroom  for  testing  situations.  The  most  common  accommodation  was  extended  time.  The  samples  of 
students  from  the  third  group  of  schools  that  used  the  new  inclusion  rules  and  offered  students 
accommodations  were  not  included  for  most  analysis  and  reporting  purposes,  although  the  results  for 
these  samples  were  studied  in  follow-up  analyses.  In  the  State  Assessments,  no  special  accommodations 
were  offered.  The  information  in  Chapters  3 and  5 applies  to  schools  and  students  in  all  of  the  sample 
types,  while  the  data  analysis  chapters  reflect  schools  and  students  in  reporting  samples  only. 

The  new  inclusion  rules  are  applied  only  when  a student  has  been  categorized  in  his  or  her  lEP  as  a 
student  with  disabilities  (SD)  or  as  a limited  English  proficient  (LEP)  student;  all  other  students  were 
asked  to  participate  in  the  assessment.  For  this  reason,  the  sample  of  students  that  was  selected  for  most 
analysis  and  reporting  purposes  for  the  main  mathematics  assessment  consisted  of  students  from  two 
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Figure  1-1 

Subsamples  of  the  Mathematics  Assessment:  1996 


GROUPS  OF 
STUDENTS 

GROUPS  OF  SCHOOLS 

Sample  Type  1 
OLD  INCLUSION  RULES 
- NO  ACCOMMODATIONS  - 

Sample  Type  2 
NEW  INCLUSION  RULES 
- NO  ACCOMMODATIONS  - 

Sample  Type  3 
NEW  INCLUSION  RULES 

NOT  SD/LEP* 

Ai 

A2 

INCLUDED  SD/LEP* 

EXCLUDED  SD/LEP* 

msmrnv:  ' 

! 

' Students  with  Disabilities/Limited  English  Proficient 

^ Results  for  students  in  subsamples  B2,  A3,  and  B3  were  not  reported  in  NAEP  1996  Mathematics:  Report  Card  for  the  Nation  and 
the  States. 

^ Students  in  subsamples  C|,  C2,  and  C3  were  not  included  in  the  assessment. 


groups:  those  who  were  not  categorized  as  SD  or  LEP  students  and  who  were  from  schools  providing  no 
accommodations  and  used  either  set  of  inclusion  rules  (Ai  and  A2  in  Figure  1-1);  and  those  who  were 
categorized  as  SD  or  LEP  students  and  who  were  from  schools  providing  no  accommodations  and  using 
only  the  old  inclusion  rules  (Bj  in  Figure  1-1).  The  advantage  of  this  reporting  sample  is  that  it  preserves 
trend  with  previous  assessments  and  it  makes  use  of  most  of  the  data  from  the  assessment.  The  main 
science  assessment  sample  did  not  include  students  from  schools  using  the  old  inclusion  rules.  The 
sample  of  students  that  was  selected  for  most  analysis  and  reporting  purposes  for  the  main  science 
assessment  consisted  of  students  from  the  schools  using  new  inclusion  rules,  but  not  providing  any 
accommodations  (A2-and  B2  in  Figure  1-1).  The  advantage  of  this  reporting  sample  is  that  it  makes  use  of 
the  most  up-to-date  inclusion  rules  and  begins  a science  trend  line. 

Special  analyses  that  used  the  national  science  and  mathematics  assessment  data  to  compare  the 
old  and  new  inclusion  rules  and  examine  the  effect  of  offering  testing  accommodations  indicated  little 
difference  in  proportions  of  students  included  in  the  assessment.  More  students  were  included  in  the 
assessment  when  they  were  offered  accommodations;  however,  a portion  of  students  who  would  have 
participated  in  the  assessment  under  standard  conditions  was  assessed  with  accommodations  when  they 
were  offered.  A result  of  this  is  that  fewer  students  were  assessed  under  standard  conditions  when 
accommodations  were  offered.  The  students  from  the  schools  offering  accommodations  were  not 
included  in  the  analyses  or  results  contributing  to  the  Mathematics  or  Science  Report  Cards,  so  they  did 
not  affect  the  measurement  of  the  1990,  1992,  and  1996  trend  for  mathematics.  The  results  from  the 
science  assessment  were  not  compared  to  those  from  previous  assessments. 

NAEP’s  design  for  1996  required  collecting  24  different  samples  in  order  to  conduct  the 
assessments.  The  various  samples  collected  and  reported  for  the  1996  assessment  are  summarized  in 
Table  1-1. 
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Table  1-1 

NAEP  1996  Student  Samples 


Sample 

Booklet 

IDs 

Mode 

Cohort 

Assessed 

Time  of 
Testing^ 

Age 

Deflnition 

Modal 

Grade 

Reporting 

Sample 

Size^ 

9[Math  Main] 

101-129,  921 

Print 

Grade  4 

1/3/96  - 3/29/96 

6,627 

13 [Math  Main] 

101-130,  921 

Print 

Grade  8 

. 1/3/96  - 3/29/96 

7,146 

17[Math  Main] 

101-130 

Print 

Grade  12 

' 1/3/96  - 3/29/96 

6,904 

4 [Math-Estimation] 

127 

Tape 

Grade  4 

1/3/96  - 3/29/96 

2,023 

8 [Math-Estimation] 

127 

Tape 

Grade  8 

1/3/96  - 3/29/96 

2,183 

12  [Math-Estimation] 

127 

Tape 

Grade  12 

1/3/96  - 3/29/96 

1,849 

4 [Math-Theme] 

128,  129 

Print 

Grade  4 

1/3/96  - 3/29/96 

3,790 

8 [Math-Theme] 

128,  129 

Print 

Grade  8 

1/3/96  - 3/29/96 

4,027 

12  [Math;Theme] 

128,  129 

Print 

Grade  12 

1/3/96  - 3/29/96 

3,735 

8 [Math-Advanced] 

130 

Print 

Grade  8 

1/3/96  - 3/29/96 

2,337 

12  [Math- Advanced] 

130 

Print 

Grade  12 

1/3/96  - 3/29/96 

2,965 

9 [Science  Main] 

201-237 

Print 

Grade  4 

1/3/96  - 3/29/96 

7,305 

13  [Science  Main] 

201-237 

Print 

Grade  8 

1/3/96  - 3/29/96 

7,774 

17  [Science  Main] 

201-240 

Print 

Grade  12 

1/3/96  - 3/29/96 

7,537 

12  [Sci-Advanced] 

238-240 

Print 

Grade  12 

1/3/96  - 3/29/96 

2,431 

13  [Sci-State] 

201-237 

Print 

Grade  8 

1/29/96  - 3/4/96 

*3 

9 [MS-LTTrend] 

91-93 

Tape 

Age  9 

1/3/96  - 3/8/96  ' 

CY 

4 

5,414 

13  [MS-LTTrend] 

91-93 

' Tape 

Age  13 

10/9/95  - 12/22/95 

CY 

8 

5,658 

17  [MS-LTTrend] 

84-85 

Tape 

Age  17 

3/11/96-5/10/96 

Not  CY 

11 

3,539 

‘ Final  makeup  sessions  for  the  winter  session  (January  3-March  29,  1996)  were  held  April  1-5,  1996. 

^The  total  number  of  students  assessed  in  the  reporting  sample  of  the  national  assessment  was  43,293  for  the  main  assessment,  29,792  for 
the  long-term  trend  assessment,  and  25,340  for  the  special  studies. 

^ Note:  consists  of  distinct  samples  in  48  jurisdictions 


LEGEND:  Math 

Mathematics 

Main 

Main  assessment,  print  administration 

Sci 

Science 

LTTrend 

Long-term  trend  assessment 

RW 

Reading  and  writing 

CY 

Calendar  year:  birthdates  in  1986  and  1982 

MS 

Mathematics  and  science 

respectively  for  ages  9 and  1 3 

Print 

Print  administration 

Not  CY 

Age  17  only:  birthdates  between  October  1 , 1978 

Tape 

Audiotape  administration 

and  September  30, 1979 

Each  row  of  Table  1-1  corresponds  to  a particular  sample  and  each  column  of  the  table  indicates  the 
following  major  features  of  that  sample: 

1 . Sample  is  the  sample  identifier.  The  first  part  of  the  sample  code  is  a number  (the  age 
class)  representing  the  student  cohort  included  in  the  sample  (note  that  this  part  of  the 
code  does  nm  indicate  whether  an  age  or  grade  sample  was  selected);  the  second  part, 
in  brackets,  denotes  the  specific  sample  type.  For  example,  9[Math  Main]  is  a main 
assessment  mathematics  sample  for  grade  4,  assessed  in  print  mode.  A full  description 
of  the  purposes  for  the  various  sample  types  is  given  below. 

2.  Booklets  gives  the  identifier  numbers  for  the  booklets  used  for  the  assessment  of  the 
particular  sample. 
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3.  Mode  indicates  the  mode  of  assessment,  which  may  be  print  or  tape.  NAEP  originally 
assessed  students  using  a tape  recorder  in  addition  to  booklets,  thus  pacing  the 
students  through  exercises  at  a fixed  rate.  In  1996,  NAEP  used  a paced  audiotape  for 
its  mathematics  and  science  long-term  trend  assessments.  However,  most  other 
assessments  in  1996  used  printed  instructions  with  the  student  expected  to  read  the 
exercises.  The  only  other  exception  was  the  1996  assessment  of  mathematics 
estimation  skills. 

4.  The  cohort  assessed  denotes  the  age,  grade,  or  age/grade  of  the  population  being 
sampled.  For  example,  grade  4 represents  students  who  are  in  the  fourth  grade;  an  age 
17  cohort  consists  of  students  (in  any  grade)  who  are  17  years  old.  Samples  for  the 
1996  main  assessments  were  selected  on  the  basis  of  grade  only.  The  traditional 
NAEP  samples  used  in  long-term  trend  estimation  were  defined  by  age  only. 

However,  the  1996  reading  and  writing  long-term  trend  assessments  were  defined  by 
being  either  of  a particular  age  or  of  the  modal  grade  for  students  of  that  age.  For 
reading  and  writing,  results  are  reported  for  grade  and  age  samples,  respectively.  The 
definitions  of  age,  and  thus  the  corresponding  grade,  have  changed  in  ways  that  are 
described  in  Section  1.2.1. 

5.  Time  of  testing  indicates  the  time  of  year  in  which  the  assessment  is  performed. 

NAEP  traditionally  assessed  9-year-olds  in  the  winter,  13-year-olds  in  the  fall,  and  17- 
year-olds  in  the  spring;  like  the  1994  main  assessment,  in  1996,  all  grades  were 
assessed  in  the  winter  (between  January  3-March  29,  1996;  final  makeup  sessions 
were  held  April  1-5,  1996). 

6.  Age  definition  is  denoted  as  calendar  year  (CY)  or  not  calendar  year  (Not  CY).  NAEP 
originally  defined  age  by  birth  within  a calendar  year  at  ages  9 and  13  but  defined  age 
17  as  being  born  between  October  1 of  one  year  and  September  30  of  the  next.  In  the 
1996  main  assessments,  no  students  were  selected  on  the  basis  of  their  age. 

7.  The  modal  grade  is  the  grade  attended  by  most  of  the  students  of  the  sampled 
age.  For  example,  if  an  age  17  sample  is  listed  as  having  a modal  grade  of  11, 
then  most  of  the  17-year-old  students,  as  defined,  are  in  the  eleventh  grade.  The 
definition  of  age  affects  the  modal  grade  of  the  sample.  All  students  sampled  for 
the  1996  main  assessments  were  in  the  grade  defined  by  the  cohort  assessed. 

8.  The  reporting  sample  size  is  the  number  of  students  in  the  sample  who  were 
actually  administered  the  assessment  and  whose  results  were  used  in  the  NAEP 
subject-area  reports. 


1.2.1  The  1996  NAEP  Samples 

The  NAEP  samples  in  1996  consisted  of  four  types:  the  samples  from  the  National 
Assessment — the  main  NAEP  samples,  the  long-term  trend  samples,  and  the  special  studies  samples — 
and  the  State  Assessment  samples. 

The  Main  NAEP  Samples.  The  main  NAEP  samples  are  labeled  in  Table  1-1  as  [Math  Main] 
and  [Science  Main].  The  samples  used  complex  spiraling  procedures  (defined  in  Section  1.5),  and  were 
intended  to  form  the  basis  for  future  assessments.  Each  sample  was  assessed  in  the  winter  period 
(January  3 through  March  29,  1996).  In  these  samples,  only  grade  populations  were  sampled,  although 
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age/grade  populations  were  assessed  in  previous  assessment  years.  The  main  NAEP  samples,  and  their 
purposes,  are  as  follows: 

[Math  Main]  are  grades  4,  8,  and  12  mathematics  assessment  samples  used  for  measuring 
mathematics  achievement  in  1996.  The  fourth-  and  eighth-grade  samples  also  provided  the 
comparison  groups  for  the  1996  State  Assessment  of  mathematics  in  grades  4 and  8.  These 
samples  used  print  administration. 

[Science  Main]  are  grades  4,  8,  and  12  science  assessment  samples  used  for  measuring  science 
achievement  in  1996.  The  eighth-grade  samples  also  provided  the  cornparison  groups  for  the 
1996  State  Assessment  in  science  in  grade  8.  These  samples  used  print  administration. 

The  Long-Term  Trend  Samples.  The  long-term  trend  samples  are  labeled  as  [RW-LTTrend]  and 
[MS-LTTrend]  in  Table  1-1.  Each  sample  was  defined  in  the  same  way  as  equivalent  samples  in  previous 
assessments  and  used  the  same  assessment  technology  as  was  used  in  those  assessments.  Therefore,  the 
long-term  trend  samples  are  directly  comparable  to  those  from  previous  assessments  and  so  can  be  used 
for  continuing  the  NAEP  long-term  trend  lines.  Because  these  samples  were  designed  to  link  the  1996 
data  with  data  from  previous  assessments,  they  are  also  referred  to  as  bridge  samples.  The  long-term 
trend  samples  and  their  purposes  are  as  follows: 

[RW-LTTrend]  are  age/grade  samples  used  for  estimating  long-term  trends  in  reading 
and  writing.  These  samples  used  assessment  booklets  identical  to  those  initially  used  in 
1984  and  subsequently  used  in  1988,  1990,  1992,  and  1994  (many  of  the  items  were  also 
used  in  pre-1984  assessments).  As  in  1984,  1988,  1990,  1992,  and  1994,  print 
administration  was  used.  These  samples  used  the  age  definitions  and  time  of  testing 
originally  used  by  NAEP  in  the  1970s  and  the  early  1980s.  The  estimates  of  reading 
achievement  from  these  samples  link  to  eight  previous  reading  assessments  (1971,  1975, 

1980,  1984,  1988,  1990,  1992,  and  1994);  the  estimates  of  writing  achievement  link  to 
five  previous  writing  assessments  (1984,  1988,  1990,  1992,  and  1994). 

[MS-LTTrend]  are  age-only  samples  used  for  estimating  long-term  trends  in  mathematics 
and  science  achievement.  These  samples  used  the  same  age  definitions  and  time  of 
testing  as  were  used  since  1969  and  used  the  same  assessment  instruments  as  were  used 
in  the  1986,  1990,  1992,  and  1994  long-term  trend  assessments  of  mathematics  and 
science.  As  in  previous  assessments,  the  administration  of  the  mathematics  and  science 
questions  was  paced  with  an  audiotape.  The  estimates  of  science  achievement  from  these 
samples  link  to  eight  previous  science  assessments  (1970,  1973,  1977,  1982,  1986,  1990, 

1992,  and  1994);  the  estimates  of  mathematics  achievement  link  to  seven  previous 
assessments  (1973,  1978,  1982,  1986,  1990,  1992,  and  1994). 

The  Special  Studies  Samples.  Three  sets  of  samples  were  collected  as  part  of  special  NAEP 
studies.  The  samples  used  special  innovative  procedures  to  allow  the  study  of  specific  aspects  of 
mathematics  and  science.  Each  sample  was  assessed  in  the  winter  period  (January  3 through  March  29, 
1996).  In  these  samples,  only  grade  populations  were  sampled.  The  special  studies  samples,  and  their 
purposes,  are  as  follows: 

[Math-Estimation]  are  samples  of  specially  selected  students  in  grades  4,  8,  and  12  who 
were  administered  mathematics  estimation  booklets  in  separate  paced-tape  sessions.  The 
students  are  representative  of  the  fourth-,  eighth-,  and  twelfth-grade  students  in  the 
nation. 
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[Math-Theme]  are  samples  of  specially  selected  students  in  grades  4,  8,  and  12  who  were 
administered  mathematics  theme  booklets.  These  samples  were  assessed  in  print 
administrations.  The  students  were  selected  to  represent  the  national  populations  of 
fourth-,  eighth-,  and  twelfth-grade  students.  The  students  in  these  samples  were  assessed 
in  separate  sessions. 

[Math-Advanced]  and  [Sci-Advanced]  are  samples  of  specially  selected  students  in  grade 
8 (for  mathematics  only)  and  grade  12  (for  mathematics  and  science)  who  received 
advanced  mathematics  and  science  booklets.  They  were  assessed  in  separate  sessions. 

The  students  were  selected  from  students  who  were  taking  advanced  courses  in 
mathematics  or  science. 

The  State  Assessment  Samples.  In  Table  1-1,  9 [Math-State],  13 [Math-State],  and  13[Sci-State] 
are  samples  of  fourth-  and  eighth-grade  public-  and  nonpublic-school  students  from  each  of  the  states  and 
jurisdictions  participating  in  the  1996  State  Assessment  of  mathematics,  and  eighth-grade  public-  and 
nonpublic-school  students  from  each  of  the  states  and  jurisdictions  participating  in  the  1996  State 
Assessments  of  science.^  The  assessment  booklets  were  the  same  print-administered  booklets  as  those 
used  for  the  matching  samples  9[Math  Main],  13[Math  Main],  and  13[Science  Main]  but  the 
administrative  procedures  varied  from  that  of  the  national  assessment  in  that  state  personnel  collected  the 
data. 


1.2.2  NAEP  Assessments  Since  1969 

Table  1-2  shows  the  subject  areas,  grades,  and  ages  assessed  since  the  NAEP  project  began  in 
1969.  As  can  be  seen,  in  addition  to  the  1996  subject  areas  of  mathematics,  science,  reading,  and  writing, 
several  other  subject  areas  have  been  assessed  over  the  years — social  studies,  U.S.  history,  civics, 
citizenship,  geography,  literature,  music,  career  development,  art,  and  computer  competence.  Many 
subject  areas  are  reassessed  periodically  to  measure  trends  over  time. 

Assessments  were  conducted  annually  through  1980,  but  budget  restrictions  since  then  have 
reduced  data  collection  to  a biennial  basis.  Since  its  inception,  NAEP  has  assessed  9-year-olds,  13-year- 
olds,  and  in-school  17-year-olds,  although  the  age  definitions  changed  in  1986  and  again  in  1988. 
Because  of  budget  restrictions,  NAEP  no  longer  routinely  assesses  out-of-school  17-year-olds  or  young 
adults.  (A  separate  assessment  of  young  adults  of  ages  21  to  25  was  conducted  in  1985  under  a separate 
grant.) 


The  table  also  indicates  that  in  1984,  NAEP  began  gathering  data  by  grade  as  well  as  by  age,  a 
practice  that  had  been  continued  in  assessments  up  to  1994;  the  1996  national  assessment  included 
gathered  data  by  grade  only.  It  should  be  noted  that  somewhat  different  age  definitions  were  used  in  the 
1984,  1986,  and  1988  assessments.  In  the  1984  assessment,  the  two  younger  ages  were  defined  on  a 
calendar-year  basis,  while  the  17-year-olds  were  defined  on  an  October  1 to  September  30  basis.  This 
resulted  in  modal  grades  of  4,  8,  and  11.  To  allow  for  age  cohorts  that  were  exactly  four  years  apart,  in 
the  1986  main  assessment  all  ages  were  defined  on  an  October  1 to  September  30  basis,  resulting  in 
modal  grades  of  3,  7,  and  11.  Special  studies  (Kaplan,  Beaton,  Johnson,  & Johnson,  1988)  were 
conducted  to  measure  the  effect  of  the  changes  in  age  definition.  Because  of  problems  encountered  in 
assessing  third  graders,  in  1988  the  ages  were  redefined  on  a calendar-year  basis,  with  the  modal  grades 
being  4,  8,  and  12.  These  were  the  age  definitions  used  in  the  1990,  1992,  and  1994  main  assessments. 


^ Fourth-grade  students  in  DDESS  and  DoDDS  schools  were  assessed  as  a separate  special  science  assessment. 
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Table  1-2 

National  Assessment  of  Educational  Progress 
Subject  Areas,  Grades,  and  Ages  Assessed:  1969-1996 


Grades/Ages  Assessed 

Assessment 

Grade 

Grade 

Age 

Grade 

Grade 

Age 

Grade  Grade 

Age 

Age 

Year 

Subiect  Area(s) 

3 

4 

9 

7 

8 

13 

11  12 

17 

170S* 

Adult 

1969-70 

Science 

X 

X 

X 

X 

X 

Writing 

X 

X 

X 

X 

X 

Citizenship 

X 

X 

X 

X 

X 

1970-71 

Reading 

X 

X 

X 

X 

X 

Literature 

X 

X 

X 

X 

X 

1971-72 

Music 

X 

X 

X 

X 

X 

Social  Studies 

X 

X 

X 

X 

X 

1972-73 

Science 

X 

X 

X 

X 

X 

Mathematics 

X 

X 

X 

X 

X 

1973-74 

Career  and  Occupational 

Development 

X 

X 

X 

X 

X 

Writing 

X 

X 

X 

X 

1974-75 

Reading 

X 

X 

X 

X 

Art 

X 

X 

X 

X 

Index  of  Basic  Skills 

X 

X 

1975-76 

Citizenship/Social  Studies 

X 

X 

X 

. X 

1976-77 

Science 

X 

X 

X 

Basic  Life  Skills^ 

X 

Sci/  RD  / Energy  / Health^ 

X 

1977-78 

Mathematics 

X 

X 

X 

Consumer  Skills^ 

X 

1978-79 

Writing,  Art,  and  Music 

X 

X 

X 

1979-80 

Reading/Literature 

X 

X 

X 

X 

Art 

X 

1981-82 

1 ' 

Science 

X 

X 

X 

Mathematics  and 

Citizenship/Social  Studies 

X 

X 

X 

1984 

Reading 

X 

X 

X 

X 

X 

X 

Writing 

X 

X 

X 

X 

X 

X 

Writing  (long-term  trend) 

X 

X 

X 

X 

X 

X 

1985 

Adult  Literacy^ 

X 

1986 

Reading 

X 

X 

X 

X 

X 

X 

Mathematics 

X 

X 

X 

X 

X 

X 

Science 

X 

X 

X 

X 

X 

X 

Computer  Competence 

X 

X 

X 

X 

X 

X 

U.S.  History^ 

X 

X 

Literature^ 

X 

X 

Reading  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Mati^matics  Gong-tom  trend) 

X 

X 

X 

X 

X 

X 

Science  (long-term  trend) 

X 

X 

X 

X 

X 

X 

' Age  17  students  who  had  dropped  out  of  school  or  had  graduated  prior  to  assessment. 

^ Small,  special-interest  assessment  conducted  on  limited  samples  at  specific  grades  or  ages. 
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Table  1-2  (continued) 

National  Assessment  of  Educational  Progress 
Subject  Areas,  Grades,  and  Ages  Assessed:  1969-1996 


Grades/Ages  Assessed 

Assessment 

Grade  Grade 

Age 

Grade  Grade 

Age 

Grade 

Grade 

Age 

Age 

Year^ 

Subject  Area(s) 

3 4 

9 

7 8 

13 

11 

12 

17 

170S‘  Adult 

1988 

Reading 

X 

X 

X 

X 

X 

X 

Writing 

X 

X 

X 

X 

X 

X 

Civics 

X 

X 

X 

X 

X 

X 

U.S.  History 

X 

X 

X 

X 

X 

X 

Document  Literacy^ 

X 

X 

X 

X 

X 

X 

Geography^ 

X 

X 

X 

X 

X 

X 

Reading  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Writing  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Mathematics  (long-ton  trend) 

X 

X 

X 

X 

Science  (long-term  trend) 

X 

X 

X 

X 

Civics  (long-term  trend) 

X 

X 

1990 

Reading 

X 

X 

X 

X 

X 

X 

Mathematics 

X 

X 

X 

X 

X 

X 

Science 

X 

X 

X 

X 

X 

X 

Reading  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Writing  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Mathematics  (long-term  trend) 

X 

X 

X 

Science  (long-term  trend) 

X 

X 

X 

Trial  State  Mathematics 

X 

1992 

Reading 

X 

X 

X 

X 

X 

X 

Writing 

X 

X 

X 

X 

X 

X 

Mathematics 

X 

X 

X 

X 

X 

X 

Reading  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Writing  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Mathematics  (long-term  trend) 

X 

X 

X 

Science  (long-term  trend) 

X 

X 

X 

Trial  State  Mathematics 

X 

X 

Trial  State  Reading 

X 

1994 

Reading 

X 

X 

X 

X 

X 

X 

U.S.  History 

X 

X 

X 

X 

X 

X 

Geography  ^ 

X 

X 

X 

X 

X 

X 

Reading  (long-term  trend)| 

X 

X 

X 

X 

X 

X 

Writing  (long-term  trend)  j 

X 

X 

X 

X 

X 

X 

Mathematics  (long-ton  trend) 

X 

X 

X 

Science  (long-term  trend) 

X 

X 

X 

Trial  State  Reading 

X 

' Age  17  students  who  had  dropped  out  of  school  or  had  graduated  prior  to  assessment. 

^ Small,  special-interest  assessment  conducted  on  limited  samples  at  specific  grades  or  ages. 

^ It  should  be  noted  that  somewhat  different  age  definitions  were  used  in  the  1984,  1986,  and  1988  assessments.  In  the  1984  assessment,  the  two 
younger  ages  were  defined  on  a calendar-year  basis,  while  the  17-year-olds  were  defined  on  an  October  1 to  September  30  basis.  This  resulted  in 
modal  grades  of  4,  8,  and  11.  To  allow  for  age  cohorts  that  were  exactly  four  years  apart,  in  the  1986  main  assessment  all  ages  were  defined  on 
an  October  1 to  September  30  basis,  resulting  in  modal  grades  of  3,  7,  and  11.  Special  studies  (Kaplan,  Beaton,  Johnson,  & Johnson,  1988)  were 
conducted  to  measure  the  effect  of  the  changes  in  age  definition.  Because  of  problems  encountered  in  assessing  third  graders,  in  1988  the  ages 
were  redefined  on  a calendar-year  basis,  with  the  modal  grades  being  4,  8,  and  12.  These  were  the  age  definitions  used  in  the  1990,  1992,  1994, 
and  1996  main  assessments. 
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Table  1-2  (continued) 

National  Assessment  of  Educational  Progress 
Subject  Areas,  Grades,  and  Ages  Assessed:  1969-1996 


Grades/Ages  Assessed 

Assessment  Grade 

Year^  Subject  Area(s)  3 

Grade 

4 

Age 

9 

Grade  Grade 
7 8 

Age 

13 

Grade 

11 

Grade 

12 

Age  Age 
17  170S‘  Adult 

1996  Mathematics 

X 

X 

X 

Science 

X 

X 

X 

Reading  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Writing  (long-term  trend) 

X 

X 

X 

X 

X 

X 

Mathematics  (long-temi  tr^) 

X 

X 

X 

Science  (long-term  trend) 

X 

X 

X 

State  Mathematics 

X 

X 

State  Science 

X 

X 

' Age  1 7 students  who  had  dropped  out  of  school  or  had  graduated  prior  to  assessment. 

•'  It  should  be  noted  that  somewhat  different  age  definitions  were  used  in  the  1984,  1986,  and  1988  assessments.  In  the  1984  assessment,  the  two 
younger  ages  were  defined  on  a calendar-year  basis,  while  the  17-year-olds  were  defined  on  an  October  1 to  September  30  basis.  This  resulted  in 
modal  grades  of  4 8 and  11.  To  allow  for  age  cohorts  that  were  exactly  four  years  apart,  in  the  1986  mam  assessment  all  ages  were  defined  on 
an  October  1 to  September  30  basis,  resulting  in  modal  grades  of  3,  7,  and  11.  Special  studies  (Kaplan,  Beaton,  Johnson,  & Johnson,  1988)  were 
conducted  to  measure  the  effect  of  the  changes  in  age  definition.  Because  of  problems  encountered  in  assessing  third  graders,  in  1988  the  ages 
were  redefined  on  a calendar-year  basis,  with  the  modal  grades  being  4,  8,  and  12.  These  were  the  age  definitions  used  in  the  1990,  1992,  1994, 
and  1996  main  assessments. 


1.3  DEVELOPMENT  OF  ASSESSMENT  OBJECTIVES,  ITEMS,  AND 
BACKGROUND  QUESTIONS 

In  1996,  NAEP  conducted  main  assessments  of  students  at  all  three  grade  levels  in  mathematics 
and  science.  These  assessments  entailed  the  generation  of  a large  number  of  cognitive  items 
items  measuring  knowledge  and  skills.  In  addition,  a large  number  of  background  and  attitude  questions 
were  asked  of  students,  and  school,  teacher  and  instructional  questions  were  asked  of  principals  and 
teachers.  Details  on  the  item  development  procedures  for  the  1996  main  assessment  are  given  in  Chapter 
2;  this  section  provides  an  overview.  (In  addition  to  the  main  assessments  and  the  special  assessments, 
long-term  trend  studies  were  conducted  in  reading,  mathematics,  science,  and  writing.  Since  the 
instruments  used  for  these  studies  consisted  entirely  of  items  used  in  previous  assessments,  no 
developmental  tasks  were  required  for  their  use  in  the  1996  assessment.) 

In  addition  to  the  cognitive  items,  several  questionnaires  were  developed:  a common  student 
Background  Questionnaire  given  to  all  assessed  students  of  a given  grade,  a School  Characteristics  and 
Policies  Questionnaire,  Teacher  Questionnaires  for  teachers  of  fourth-  and  eighth-grade  students  in 
mathematics  and  science  and  for  teachers  of  twelfth-grade  students  who  were  assessed  in  advanced 
mathematics,  and  an  SD/LEP  Student  Questionnaire.  Each  of  these  questionnaires  was  developed 
through  a broad-based  consensus  process. 

All  items  in  the  assessment  underwent  extensive  reviews  by  subject  area  and  measurement 
specialists,  as  well  as  careful  scrutiny  to  eliminate  any  potential  bias  or  lack  of  sensitivity  to  any  group. 
Further,  the  items  were  field  tested  on  a representative  group  of  students.  Based  on  the  results  of  the  field 
test,  items  were  revised  or  modified  as  necessary  and  then  again  reviewed  for  bias.  With  the  help  of  staff 
and  outside  reviewers,  the  Instrument  Development  Panels  selected  the  items  to  include  in  the 
assessment.  After  the  items  were  selected  and  formed  into  the  final  groupings  or  blocks  of  items,  they 
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were  carefully  reviewed  by  the  National  Center  for  Education  Statistics  (NCES),  the  Office  of 
Management  and  Budget  (0MB),  and  the  National  Assessment  Governing  Board  (NAGB). 

The  assessment  instruments  included  multiple-choice  items,  constructed-response  items  scored 
dichotomously,  constructed-response  items  scored  polytomously,  and  cluster  items  in  both  mathematics 
and  science.  The  constructed-response  items  were  professionally  scored  as  described  in  Chapter  7. 


1.4  THE  1996  SAMPLE  DESIGN 

The  sample  for  the  1996  NAEP  assessment  was  selected  using  a complex  three-stage  sample 
design.  The  three-stage  sample  design  includes  (1)  the  sampling  of  students  from  (2)  selected  schools 
within  (3)  94  selected  geographic  areas,  called  primary  sampling  units  (PSUs),  across  the  United  States. 
The  1996  sample  design  differed  from  previous  years  due  to  oversampling  of  SD/LEP  students.  With  the 
inclusion  of  the  different  inclusion  rules  and  the  availability  of  accommodations,  the  sample  deisign  was 
similar  to  that  used  in  1986,  1988,  1990,  1992,  and  1994  and  is  described  in  detail  by  Westat,  Inc.,  the 
firm  contracted  by  NCES  to  select  the  sample,  in  The  1996  NAEP  Sampling  and  Weighting  Report 
(Wallace  & Rust,  1999).  The  following  sections  provide  an  overview  of  the  five  steps  used  to  draw 
NAEP  samples  using  the  three-stage  sample  design.  Further  details  are  given  in  Chapter  3.  Steps  3 and  4 
describe  the  assignment  of  sample  types  and  assessment  sessions  to  the  second  sampling  unit  schools. 
Steps  5a  through  5c  contain  procedures  for  the  collection  of  data  for  SD/LEP  students,  teachers,  and 
schools. 


O Step  1:  Primary  Sampling  Units 

In  the  first  stage  of  sampling,  the  United  States  (the  50  states  and  the  District  of  Columbia)  was 
divided  into  geographic  primary  sampling  units  (PSUs).  Each  PSU  met  a minimum  size  requirement  and 
generally  comprised  either  a consolidated  metropolitan  statistical  area  (CMSA),  a metropolitan  statistical 
area  (MSA),  a single  county,  or  a group  of  contiguous  counties.  The  PSUs  were  classified  into  four 
regions  (Northeast,  Southeast,  Central,  West),  each  containing  about  one-fourth  of  the  U.S.  population. 

In  each  region,  PSUs  were  additionally  classified  as  MSA  or  nonMSA.  This  resulted  in  eight 
subuniverses  of  PSUs. 

Ninety-four  of  the  PSUs  were  selected  for  the  1996  main  assessment.  Twenty-two  PSUs  were 
designated  as  certainty  units  because  of  their  size,  and  were  included  in  the  sample  with  certainty.  The 
remaining  smaller  PSUs  were  not  guaranteed  to  be  selected  and  were  accordingly  designated  as 
noncertainty  PSUs.  Within  each  major  stratum  (subuniverse),  further  stratification  was  achieved  by 
ordering  the  noncertainty  PSUs  according  to  several  additional  socioeconomic  characteristics.  Seventy- 
two  PSUs  were  selected,  one  per  stratum  from  each  of  the  noncertainty  strata,  with  probability 
proportional  to  size  (total  population  from  the  1990  census).  To  enlarge  the  samples  of  Black  and 
Hispanic  students,  thereby  enhancing  the  reliability  of  estimates  for  these  groups,  PSUs  from  the  high- 
minority  subuniverses  were  sampled  at  twice  the  rate  of  those  from  the  other  subuniverses.  This  was 
achieved  by  creating  smaller  strata  within  the  high-minority  subuniverses. 

For  the  long-term  trend  samples,  52  PSUs  were  selected:  10  PSUs  were  selected  with  certainty; 
six  additional  PSUs  were  selected  from  the  12  remaining  main  sample  certainty  PSUs;  and  36  PSUs  were 
selected  from  the  72  noncertainty  strata  independently  of  PSU  selections  for  the  main  samples. 
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O Step  2:  Selection  of  Schools 

In  the  second  stage  of  sampling  for  the  main  assessments,  the  public  schools  (including  Bureau 
of  Indian  Affairs  (BIA)  schools  and  Department  of  Defense  Education  Activity  (DoDEA)  schools)  and 
nonpublic  schools  (including  Catholic  schools)  within  each  of  the  selected  PSUs  were  listed  according  to 
the  grade  ranges  associated  with  the  three  age  classes.  An  independent  sample  of  schools  was  selected 
separately  for  each  of  the  grades  so  that  some  schools  were  selected  for  assessment  of  two  grades,  and  a 
few  were  selected  for  all  three.  Schools  within  each  PSU  were  selected  (without  replacement)  with 
probabilities  proportional  to  assigned  measures  of  size  with  oversampling  of  nonpublic  schools  and  of 
schools  with  high  minority  enrollment.  Overall  probabilities  of  selection  for  high-minority  schools  were 
twice  those  for  other  schools  while  the  probabilities  of  selection  of  nonpublic  schools  were  triple  those  of 
low-minority  public  schools  of  the  same  size.  The  increased  probabilities  of  selection  enlarged  the 
samples  of  Black  and  Hispanic  students  and  the  samples  of  students  from  nonpublic  schools,  thereby 
enhancing  the  reliability  of  estimates  for  these  groups.  Details  of  the  probabilities  used  for  school 
selection  appear  in  Chapter  3. 

The  samples  of  schools  for  the  long-term  trend  assessments  were  drawn  in  a manner  very  similar 
to  that  used  for  the  main  assessments.  The  chief  difference  in  the  two  samples  was  that  nonpublic  schools 
and  schools  with  high  minority  enrollment  were  not  oversampled  for  the  long-term  trend  assessments. 
Schools  were  not  selected  for  both  main  and  long-term  trend  assessments  at  the  same  age/grade. 

For  the  main  samples,  the  overall  school  cooperation  rate  was  86  percent  for  grade  4,  83  percent 
for  grade  8,  and  79  percent  for  grade  12.  For  the  long-term  trend  samples,  the  overall  school  cooperation 
rate  was  85  percent  for  age  class  9,  84  percent  for  age  class  13,  and  81  percent  for  age  class  17.  In  certain 
instances,  refusing  schools  were  replaced  by  substitutes  according  to  the  rules  indicated  in  Chapter  3. 


O Step  3:  Assigning  Sample  Type  to  Schools 

In  order  to  determine  the  effect  of  using  different  criteria  for  excluding  students  from  the 
assessment,  three  different  sample  types  were  assigned  to  the  schools  selected  for  the  main  assessment. 
In  sample  type  1 schools,  the  inclusion  criteria  for  the  main  samples  were  identical  to  those  used  in  1990 
and  1992.  In  sample  type  2 schools,  new  1996  inclusion  criteria  were  used.  In  sample  type  3 schools,  the 
new  1996  inclusion  criteria  were  used  and  accommodations  were  offered  to  SD/LEP  students.  More 
detailed  information  on  assigning  sample  type  to  schools  is  provided  in  Chapter  3. 


O Step  4:  Assigning  Assessment  Sessions  to  Schools 

Sessions  were  assigned  to  the  selected  schools  found  to  be  in-scope  at  the  time  of  session 
assignment,  as  described  in  Chapter  3.  Sessions  were  assigned  to  schools  with  three  aims  in  mind.  The 
first  was  to  distribute  students  to  the  different  session  types  across  the  whole  sample  for  each  age  class  so 
that  the  target  numbers  of  assessed  students  would  be  achieved  (in  each  sample  type  separately  in  the 
main  assessments).  The  second  was  to  maximize  the  number  of  different  session  types  that  were 
administered  within  a given  selected  school,  without  creating  unduly  small  sessions.  The  third  was  to 
give  each  student  an  equal  chance  of  being  selected  for  a given  session  type  regardless  of  the  number  of 
sessions  conducted  in  the  school. 
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Step  5:  Sampling  Students 


In  the  third  stage  of  sampling,  a consolidated  list  was  prepared  for  each  school  of  all  grade-  and 
age-eligible  students  (for  long-term  trend)  or  all  grade-eligible  students  (for  the  main  assessments)  for  the 
age  class  for  which  the  school  was  selected.  To  provide  the  target  sample  size,  a systematic  selection  of 
eligible  students  was  made  from  this  list,  if  necessary.  In  small-  and  medium-sized  schools  all  eligible 
students  were  in  the  sample.  For  schools  assigned  to  more  than  a single  session  type,  students  were 
assigned  by  Westat  district  supervisors  to  one  of  the  various  session  types  (audiotape  or  print 
administration)  using  specified  procedures.  No  student  was  assigned  to  more  than  one  session. 

Step  5a:  Excluded  Students 

Despite  NAEP’s  goal  to  assess  all  selected  students,  certain  selected  students 
were  judged  by  school  authorities  as  being  incapable  of  participating  meaningfully  in  the 
assessment.  For  each  student  who  was  excluded,  school  staff  who  had  knowledge  of  the 
student’s  capabilities  completed  an  SD/LEP  student  questionnaire,  listing  the  reason  for 
exclusion  and  providing  some  background  information.  For  each  SD/LEP  student  who 
was  included  in  the  assessment,  school  staff  also  completed  an  SD/LEP  student 
questionnaire. 

Specific  guidelines  for  exclusion  were  provided  for  all  samples  in  the  1996 
assessment.  However,  somewhat  different  criteria  were  used  for  the  long-term  trend 
samples  than  for  the  main  assessment  samples.  In  addition,  the  inclusion  criteria  for  the 
main  samples  differed  by  sample  type. 

The  exclusion  guidelines  for  the  long-term  trend  samples  were  the  same  as  those 
used  in  previous  assessments.  Three  types  of  students  could  be  excluded  under  these 
guidelines — non-English  speaking  students,  students  with  mental  retardation  who  are 
educable  but  who  were  judged  incapable  of  meaningfully  responding  to  exercises 
appropriate  to  their  age  level,  and  students  so  functionally  disabled  that  they  could  not 
perform  in  the  NAEP  assessment  situation. 

As  stated  previously,  for  the  main  samples,  the  procedures  for  assessing  students 
with  disabilities  (SD)  and  students  of  limited  English  proficiency  (LEP)  varied  by 
sample  type.  The  exclusion  procedure  used  in  sample  type  1 differed  somewhat  from  that 
used  in  sample  types  2 and  3.  In  sample  type  1 schools,  the  inclusion  criteria  for  the  main 
samples  were  identical  to  those  used  in  1990  and  1992.  These  criteria  were  intended  to 
be  somewhat  more  rigorously  defined  than  those  used  in  the  long-term  trend  samples.  In 
sample  type  2 schools,  new  1996  inclusion  criteria  were  used.  In  sample  type  3 schools, 
the  new  1996  inclusion  criteria  were  used  and  accommodations  were  offered  to  SD/LEP 
students.  The  new  inclusion  criteria  was  developed  to  more  closely  match  the  procedures 
used  by  many  states  and  school  districts  in  testing  situations.  Both  sets  of  the  inclusion 
rules  are  presented  in  Chapter  5. 

Step  5b:  Sampling  Teachers 

Teachers  of  fourth-  and  eighth-grade  students  assessed  in  mathematics  and 
science  and  twelfth-grade  students  assessed  in  advanced  mathematics  were  identified  and 
asked  to  complete  a questionnaire  (described  in  Chapter  2)  about  their  background  and 
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experiences  and  about  instructional  practices,  by  class,  for  any  classes  containing 
assessed  students. 

Step  5c:  The  School  Characteristics  and  Policies  Questionnaires 

A School  Characteristics  and  Policies  Questionnaire  was  mailed  to  every 
sampled  school  by  Westat  before  the  assessment  for  completion  by  the  principal  or 
school  administrator.  The  Westat  supervisor  then  collected  the  questionnaires  and 
returned  them  to  ETS.  The  School  Characteristics  and  Policies  Questionnaire  is 
described  in  Chapter  2. 


1.5  ASSESSMENT  INSTRUMENTS 

Four  types  of  instruments  were  used  in  the  1996  assessment:  student  assessment  booklets  (which 
included  the  student  common  Background  Questionnaire  as  well  as  cognitive  items),  SD/LEP  Student 
Questionnaires,  Teacher  Questionnaires,  and  a School  Characteristics  and  Policies  Questionnaire.  This 
section  provides  an  overview  of  these  instruments;  more  detailed  information  can  be  found  in  Chapter  4. 

The  student  common  Background  Questionnaires  were  completed  by  the  students  participating  in 
the  1996  assessment.  These  questionnaires  included  questions  about  the  students’  race/ethnicity,  parental 
education  levels,  and  other  background  variables  specified  by  NCES  and  a committee  of  survey,  content, 
and  education  experts.  These  questionnaires  appeared  at  the  beginning  of  some  student  assessment 
booklets  and  at  the  end  of  others.  The  student  assessment  booklets  also  included  subject-related 
background  questions  about  instructional  opportunity,  and  interest  in  and  attitudes  towards  the  subject 
area. 


1.5.1  Student  Assessment  Booklets — Main  Assessment 

1.5.1.1  Student  Assessment  Booklets — Main  Assessment — Mathematics 

Each  student  assessed  in  mathematics  received  a booklet  containing  a set  of  general  background 
questions,  content  questions,  subject-specific  background  questions,  and  questions  about  his  or  her 
motivation  and  familiarity  with  the  assessment  materials.  The  content  questions  were  assembled  into 
sections  or  blocks.  Students  in  the  main  assessment  were  given  three  15-minute  blocks.  Those  sampled 
for  the  theme  assessment  completed  one  15 -minute  block  and  one  30- minute  block.  Those  sampled  for 
the  advanced  study  at  grade  8 completed  three  20-minute  blocks;  at  grade  12  advanced  sample  students 
completed  three  30-minute  blocks.  Students  in  the  estimation  sample  completed  one  15-minute  block 
from  the  main  assessment  and  two  paced-tape  sections.  The  overall  assessment  time  for  each  student  was 
approximately  63  minutes. 

The  assembly  of  blocks  into  booklets  for  the  main  assessment  and  their  subsequent  assignment  to 
sampled  students  was  determined  by  a balanced  incomplete  block  (BIB)  design  with  spiraled 
administration.  The  student  booklets  contained  two  five-minute  background  sections,  a one-minute 
background  section,  and  three  15-minute  blocks  of  items  according  to  a BIB  design. 

The  BIB  design  for  the  1996  national  mathematics  assessment  was  focused  by  subject  area,  so 
that  students  received  booklets  containing  only  blocks  of  mathematics  questions  (not  science).  The  BIB 
design  also  balances  the  order  of  presentation  of  the  15-minute  blocks  of  items — every  15-minute  block 
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appears  as  the  first  cognitive  block  in  two  booklets,  as  the  second  cognitive  block  in  two  other  booklets, 
and  as  the  third  cognitive  block  in  another  two  booklets. 

The  design  used  in  1996  required  that  13  blocks  of  mathematics  items  at  each  grade  be  assembled 
into  26  booklets.  Theme  blocks  were  placed  in  two  other  booklets,  and  estimation  blocks  in  one  other 
booklet.  At  grades  8 and  12,  the  advanced  study  was  placed  in  one  additional  booklet.  Once  assembled, 
the  main  assessment  booklets  were  then  spiraled  and  bundled.  Spiraling  involves  interweaving  the 
booklets  in  a systematic  sequence  so  that  each  booklet  appears  an  appropriate  number  of  times  in  the 
sample.  The  bundles  were  designed  so  that  each  booklet  would  appear  equally  often  in  each  position  in  a 
bundle. 


The  final  step  in  the  BIB-spiraling  procedure  was  the  assigning  of  the  booklets  to  the  assessed 
students.  The  students  within  an  assessment  session  were  assigned  booklets  in  the  order  in  which  the 
booklets  were  bundled.  Thus,  most  students  in  an  assessment  session  received  different  booklets.  In  the 
assessment  design,  representative  and  randomly  equivalent  samples  of  students  responded  to  each  item  at 
a given  grade  level. 

Chapter  4 provides  more  detail  on  the  contents  of  the  mathematics  instruments. 


1.5.1.2  Student  Assessment  Booklets — Main  Assessment — Science 

Each  student  assessed  in  science  received  a booklet  containing  general  background  questions, 
content  questions,  subject-specific  background  questions,  and  questions  about  his  or  her  motivation  and 
familiarity  with  the  assessment  materials.  The  content  questions  were  assembled  into  sections  or  blocks. 
Students  in  the  main  assessment  were  given  three  20-minute  blocks  at  grade  4,  and  three  30-minute 
blocks  at  grades  8 and  12.  The  last  block  in  every  book  was  a hands-on  block.  Those  sampled  for  the 
advanced  study  at  grade  12  completed  four  30-minute  blocks.  The  overall  assessment  time  for  each 
student  was,  on  average,  120  minutes. 

The  assembly  of  blocks  into  booklets  for  the  main  assessment  and  their  subsequent  assignment  to 
sampled  students  was  determined  by  a complex  design  with  spiraled  administration.  The  student  booklets 
contained  two  five-minute  background  sections,  a one-minute  background  section,  and  three  blocks  of 
items. 


The  design  for  the  1996  national  assessment  was  focused  by  subject  area,  so  that  students 
received  booklets  containing  only  blocks  of  science  questions  (not  mathematics).  The  design  also 
balances  the  order  of  presentation  of  the  blocks  of  items,  except  for  the  hands-on  blocks,  which  always 
appear  in  position  three  of  a booklet.  All  other  blocks  appear  an  equal  number  of  times  in  position  one 
and  position  two.  Further,  the  design  was  set  up  to  ensure  that  no  student  answered  more  than  one  theme- 
based  block  (though  some  students  did  not  receive  any).  This  design  allows  for  some  balancing  of  the 
impact  of  context  and  fatigue  effects  to  be  measured  and  reported,  but  makes  allowance  for  the 
difficulties  and  disruption  of  administering  hands-on  blocks.  It  also  takes  into  account  the  limited  breadth 
of  content  coverage  included  in  the  theme  blocks. 

The  design  used  in  1996  required  that  fifteen  blocks  of  mathematics  items  at  each  grade  be 
assembled  into  37  booklets.  At  grade  12,  the  advanced  study  was  composed  of  three  additional  booklets. 
Once  assembled,  the  main  assessment  booklets  were  then  spiraled  and  bundled.  Spiraling  involves 
interweaving  the  booklets  in  a systematic  sequence  so  that  each  booklet  appears  an  appropriate  number 
of  times  in  the  sample.  The  bundles  were  designed  so  that  each  booklet  would  appear  equally  often  in 
each  position  in  a bundle. 


The  final  step  in  the  spiraling  procedure  was  the  assigning  of  the  booklets  to  the  assessed 
students.  The  students  within  an  assessment  session  were  assigned  booklets  in  the  order  in  which  the 
booklets  were  bundled.  Thus,  most  students  in  an  assessment  session  received  different  booklets.  In  the 
assessment  design,  representative  and  randomly  equivalent  samples  of  about  2,000  students  responded  to  each 
item  at  a given  grade  level. 

Chapter  4 provides  more  detail  on  the  contents  of  the  science  instruments. 


1.5.2  Student  Assessment  Booklets — Long-Term  Trend  Samples 

There  were  two  distinct  long-term  trend  samples  in  the  1996  assessment,  each  of  which  required 
reprinting  booklets  used  in  previous  assessments: 

Reading-Writing  Long-Term  Trend:  Six  booklets  were  used  at  each  of  the  three  age/grades  for 
the  purposes  of  measuring  long-term  trends  in  reading  and  writing.  These  booklets  were  identical  to 
booklets  used  in  the  1984  main  assessments  of  reading  and  writing  and  in  the  1988,  1990,  1992,  and  1994 
long-term  trend  assessments  of  those  subjects.  Each  booklet  consisted  of  a common  background  block  in 
the  beginning  of  each  booklet  and  three  cognitive  blocks,  either  two  reading  and  one  writing  or  one 
reading  and  two  writing.  All  cognitive  blocks  also  contained  subject-related  background  questions.  The 
booklets  were  administered  without  audiotape  and  were  spiraled  together  for  administration. 

Mathematics-Science  Long-Term  Trend'.  These  instruments  were  used  for  the  measurement  of 
mathematics  and  science  and  were  identical  to  booklets  administered  in  1990,  1992  and  1994.  These 
booklets  contained  15-minute  blocks  of  mathematics  and  science  items;  each  mathematics  block  and  each 
science  block  was  administered  using  audiotape  pacing.  (At  the  younger  two  ages,  the  booklets  also 
contain  a block  of  reading  items,  which  was  print-administered.)  There  were  three  booklets  each  at  age  9 
and  age  13  and  two  booklets  at  age  17.  The  common  background  questions  appeared  at  the  beginning  of 
each  booklet.  Combined,  the  booklets  at  an  age  contain  three  blocks  of  mathematics  items  and  three 
blocks  of  science  items.  Because  of  the  audiotape  pacing,  each  booklet  was  administered  in  a separate 
session. 


1.5.3  Other  Instruments 

Besides  the  student  assessment  booklets,  other  instruments  provided  data  relating  to  the 
assessment: 

The  SD/LEP  Student  Questionnaires  were  completed  by  the  teachers  of  those  students  who 
were  selected  to  participate  in  the  assessment  sample  who  had  disabilities  (SD)  or  were  classified  as 
Limited  English  Proficient  (LEP).  The  questionnaires  were  completed  for  all  SD  or  LEP  students, 
whether  or  not  they  actually  participated  in  the  assessment.  The  questionnaires  asked  about  the  nature  of 
the  student’s  disability  and  the  special  programs  in  which  the  student  participated.  The  response  rates  for 
this  questionnaire  ranged  from  92  to  95  percent  for  the  different  student  samples.  The  criteria  used  for 
excluding  students  are  described  in  Chapter  5. 

Teacher  Questionnaires  were  administered  to  the  teachers  of  fourth-  and  eighth-grade  students 
assessed  in  mathematics  and  science  and  teachers  of  twelfth-grade  students  who  were  assessed  in 
advanced  mathematics.  The  Teacher  Questionnaire  included  a general  section  that  contained  questions 
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about  the  teacher’s  background  and  experience.  The  rest  of  the  questionnaire  contained  questions  about 
instructional  practices,  by  class,  for  any  classes  containing  assessed  students.  The  response  rates  ranged 
from  95  to  100  percent  for  the  different  student  samples. 

School  Characteristics  and  Policies  Questionnaires  were  completed  by  school  principals  or 
their  representatives,  who  provided  information  about  school  administration,  staffing  patterns,  special 
programs,  subject  requirements,  and  school  resources.  The  response  rates  for  the  different  student 
samples  ranged  from  92  to  95  percent. 


1.6  FIELD  OPERATIONS  AND  DATA  COLLECTION 

Field  operations  and  data  collection  for  the  1996  assessment  were  the  responsibility  of  Westat, 
Inc.,  and  are  documented  in  Chapter  5.  The  field  operation  was  conducted  by  a staff  at  Westat’s  home 
office  and  a larger  staff  in  the  field.  The  Westat  home-office  staff  coordinated  all  activities  related  to 
field  operations  and  managed  materials  distribution  and  home-office  receipt  of  assessment  reporting 
forms.  The  field  staff  consisted  of  area  supervisors,  assessment  supervisors,  and  exercise  administrators. 
The  assessment  supervisors,  who  were  trained  by  Westat,  were  each  responsible  for  the  assessment 
activities  in  one  or  more  PSUs.  Although  ETS  made  initial  contact  with  participating  school  districts, 
each  assessment  supervisor  was  primarily  responsible  for  making  follow-up  contacts  with  these  districts, 
recruiting  and  training  exercise  administrators  to  work  with  them  in  administering  the  assessment 
sessions,  arranging  the  assessment  sessions,  and  selecting  the  sample  of  students  to  be  assessed  within 
each  school.  The  assessment  supervisors  and  the  exercise  administrators  administered  the  assessments, 
filled  out  the  necessary  forms,  performed  process  control,  and  shipped  the  assessment  booklets  and  forms 
to  National  Computer  Systems  (NCS),  the  subcontractor  responsible  for  processing  NAEP  materials  and 
data. 


Gaining  school  cooperation  was  the  joint  responsibility  of  Westat  and  ETS.  ETS  made  the 
preliminary  contacts  preparatory  to  obtaining  school  cooperation  by  first  contacting  the  Chief  State 
School  Officers,  informing  them  that  schools  within  their  states  had  been  selected  for  the  assessment  and, 
in  a later  letter,  listing  the  selected  schools  and  districts.  Later  mailings  were  sent  to  superintendents  of 
public  schools  and  parochial  schools  and  principals  of  other  nonpublic  schools  for  all  schools  selected  in 
the  assessment.  These  materials  provided  an  explanation  of  NAEP,  a list  of  the  selected  schools  in  the 
official’s  jurisdiction,  and  a cover  letter  explaining  that  a Westat  district  supervisor  would  contact  them 
to  set  up  an  introductory  meeting.  Westat  district  supervisors  then  scheduled  and  conducted  introductory 
meetings  (both  by  telephone  and  in  person),  worked  with  the  schools  to  schedule  the  assessments,  and, 
with  the  exercise  administrators,  conducted  the  assessments.  The  overall  participation  rate  of  schools 
originally  selected  in  the  1996  assessments  was  83  percent  for  the  main  samples  and  84  percent  for  the 
long-term  trend  samples.  Further  detail  on  school  participation  rates  is  given  in  Chapter  3. 

The  main  assessment  sessions  were  conducted  between  January  3 and  March  29,  1996,  at  all 
three  grade  levels.  The  age  9/grade  4 long-term  trend  assessments  were  carried  out  between  January  3 
and  March  8,  1996;  the  age  17/grade  11  long-term  trend  samples  were  conducted  between  March  11  and 
May  10,  1996.  The  age  13/grade  8 long-term  trend  assessments  were  carried  out  between  October  9 and 
December  22,  1995.  When  the  main  assessments  of  the  long-term  trend  subjects  were  first  collected  in 
1987  and  1986,  studies  were  completed  to  take  into  account  the  difference  in  assessment  time  across  the 
samples  (Kaplan,  Beaton,  Johnson,  & Johnson,  1988). 

Two  special  studies  that  required  additional  steps  in  the  sampling  process  were  included  in  the 
1996  main  assessment.  One  of  these  special  studies  involved  students  who  were  eligible  for  advanced 
mathematics  or  science  sessions.  Advanced  sessions  were  only  available  to  designated  students  at  grade  8 
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in  mathematics  and  at  grade  12  in  mathematics  and  science.  Further  details  on  the  advanced  sessions  is 
provided  in  Chapters  3 and  5. 

The  other  special  study  involved  applying  two  versions  of  the  SD/LEP  “inclusion”  criteria  for 
NAEP  assessments  and,  in  some  schools,  offering  accommodations  for  testing  students  designated  as 
SD/LEP.  In  the  study,  the  school  sample  was  divided  into  three  subsamples:  SI  (sample  type  1),  S2 
(sample  type  2),  and  S3  (sample  type  3).  The  purpose  of  these  subsamples  was  to  collect  data  under  the 
same  conditions  as  previous  assessments  in  order  to  maintain  trend  in  mathematics  within  NAEP; 
evaluate  the  impact  of  a revised,  more  specific  set  of  inclusion  criteria;  and  evaluate  the  combined  effect 
of  the  new  criteria  and  the  use  of  accommodations  for  testing  students.  Further  details  on  this  special 
study  are  provided  in  Chapters  3 and  5.  Results  for  the  study  appear  in  the  SD/LEP  report  along  with  a 
technical  procedural  appendix  describing  the  special  analyses  completed  for  the  study. 

An  automated  management  system  tracked  and  recorded  the  progress  of  field  work  throughout 
the  1996  assessment  period.  In  addition,  progress  was  constantly  monitored  through  telephone  reports 
held  between  the  area  supervisors  and  the  assessment  supervisors  and  between  the  area  supervisors  and 
the  home  office  staff. 

Both  Westat  and  ETS  participated  in  the  quality  control  of  the  field  administration,  which 
involved  on-site  visits  by  Westat  and  ETS  staff  to  verify  the  sampling  of  the  students  and  to  observe  the 
conduct  of  the  assessment  by  the  supervisors  and  the  exercise  administrators. 


1.7  MATERIALS  AND  DATA  PROCESSING 

After  completing  an  assessment  session,  Westat  field  supervisors  and  exercise  administrators 
shipped  the  assessment  booklets  and  forms  from  the  field  to  National  Computer  Systems  for  entry  into 
computer  files,  professional  scoring,  and  creating  the  data  files  for  transmittal  to  ETS.  Careful  checking 
assured  that  all  data  from  the  field  were  received.  More  than  134,000  booklets  and  questionnaires  were 
received  and  processed  for  the  national  portion  of  the  1996  assessment.  The  extensive  processing  of  these' 
data  is  detailed  in  Chapter  6. 

The  student  data  were  transcribed  into  machine-readable  form  by  scanning  the  student 
instruments  with  an  optical  scanning  machine.  An  intelligent  data  entry  system  was  used  for  resolution  of 
the  scanned  data,  the  entry  of  documents  rejected  by  the  scanning  machine,  and  the  entry  of  information 
from  the  questionnaires.  Additionally,  each  piece  of  input  data  was  checked  to  verify  that  it  was  of  an 
acceptable  type,  that  it  was  within  a specified  range  or  ranges  of  values,  and  that  it  was  consistent  with 
other  data  values.  The  entry  and  editing  of  materials  is  discussed  in  Chapter  6. 


1.8  PROFESSIONAL  SCORING 

Items  requiring  a written  response  from  the  student  (constructed-response  items)  were  included 
in  the  main  and  state  assessments  in  mathematics  and  science  and  in  the  long-term  trend  assessments  in 
reading,  mathematics,  and  writing.  More  than  nine  million  constructed  responses  were  read  and  marked 
by  the  professional  scoring  staff  for  the  national  and  state  portions  of  the  1996  assessment. 

Image  processing  and  scoring  were  again  used  in  1996.  Images  of  students’  responses  to  the 
constructed-response  items  were  scanned  into  computerized  form,  then  scored  online  by  professional 
raters. 
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Chapter  7 describes  the  professional  scoring  operation,  including  an  overview  of  the  scoring 
guides,  the  training  procedures,  and  the  scoring  process  for  each  subject  area. 

1.9  CREATION  OF  THE  DATABASE 

Before  analyses  could  begin,  the  student  response  data,  school,  teacher,  and  SD/LEP  student 
questionnaire  data,  and  all  sampling  weights  had  to  be  integrated  into  a coherent  and  comprehensive 
database.  This  database,  which  was  used  for  all  analyses,  was  also  the  source  for  the  creation  of  two 
NAEP  database  products — the  item  information  database  and  the  secondary-use  data  files.  Secondary-use 
data  files  include  sample  control  statement  files  for  SAS  and  SPSS  statistical  systems  and  the  NAEP 
Data  on  Disk  product  suite.  The  Data  on  Disk  products,  including  a complete  set  of  secondary-use  data 
files  on  CD-ROM,  PC-based  NAEP  data  extraction  software,  and  NAEP  analysis  modules,  make 
secondary  use  of  NAEP  data  much  easier  than  it  has  been  in  the  past.  The  quality  of  the  data  resulting 
from  the  complete  data  entry  system,  from  the  actual  instruments  collected  in  the  field  to  the  final 
machine-readable  database  used  in  analysis,  was  verified  by  selecting  field  instruments  at  random  and 
performing  a character-by-character  comparison  of  these  instruments  with  their  representations  in  the 
final  database.  Chapter  8 provides  details  on  the  database,  quality  control  activities,  and  database 
products. 


Chapter  2 


DEVELOPING  THE  NAEP  OBJECTIVES,  ITEMS, 
AND  BACKGROUND  QUESTIONS  FOR  THE 
1996  ASSESSMENTS  OF  MATHEMATICS  AND  SCIENCE 

Stephen  Lazer 
Educational  Testing  Service 


2.1  INTRODUCTION 

In  1996,  the  main  NAEP  assessments  were  conducted  in  mathematics  and  science.^  Long-term 
trend  assessments  were  also  conducted  in  reading,  writing,  mathematics,  and  science;  these  assessments 
are  composed  of  instruments  identical  to  those  used  in  previous  years.  Additional  data  were  gathered 
under  the  auspices  of  the  State  Assessment  program,  which  in  1996  assessed  mathematics  at  grades  4 and 
8 using  representative  samples  of  publicr  and  nonpublic-school  students  in  44  states^,  the  District  of 
Columbia,  Department  of  Defense  Domestic  Dependent  Elementary  and  Secondary  Schools  (DDESS), 
Department  of  Defense  Dependents  Schools  (DoDDS),  and  Guam.  The  1996  State  Assessment  program 
also  assessed  science  at  grade  8 using  representative  samples  of  public-  and  nonpublic-school  students  in 
43  states,  the  District  of  Columbia,  DDESS,  DoDDS,  and  Guam;  DDESS  and  DoDDS  students  were  also 
assessed  in  science  at  grade  4.  A summary  of  the  main  assessment  subject  areas  follows: 

Mathematics:  A mathematics  assessment  was  administered  to  national  samples  at  grades 
4,  8,  and  12.  This  assessment  was  designed  around  the  measurement  of  five  mathematics 
content  areas,  and  continued  a trend  line  begun  in  1990.  In  other  words,  many  of  the 
assessment  questions  were  used  in  the  1990  and  1992  NAEP  assessments;  others  were 
newly  developed  for  1996.  A mixture  of  multiple-choice,  short  constructed-response,  and 
extended  constructed-response  questions  made  up  the  assessment;  in  aggregate,  well 
over  half  of  the  student  assessment  time  was  spent  answering  constructed-response 
questions.  On  some  portion  of  the  assessment,  students  were  required  to  use  calculators 
and  other  hands-on  materials.  In  addition  to  the  instruments  used  to  generate  the  main 
reporting  scales,  three  supplemental  mathematics  surveys  were  conducted  at  the  national 
level.  At  each  grade,  special  instruments  were  administered  to  representative  national 
samples  that  were  designed  to  measure  the  estimation  skills  of  students.  Second,  separate 
samples  at  all  grades  were  given  “thematic”  instruments  constructed  to  measure  the 
ability  of  students  to  solve  in-depth  mathematics  problems.  Finally,  at  grades  eight  and 
twelve  a special  study  was  conducted.  In  this  study,  students  with  advanced  mathematics 
training  were  administered  special  assessnient  booklets  whose  contents  were  more 
advanced  than  those  of  the  main  assessment. 

Science:  A science  assessment  was  administered  to  national  samples  at  grades  4,  8,  and 
12.  The  assessment  measured  a broad  range  of  science-education  outcomes.  Because  the 
1996  science  assessment  was  based  on  a new  framework,  it  represents  the  beginning  of  a 


‘ Stephen  Lazer  manages  assessment  development  activities  for  the  NAEP  program  at  ETS. 

^ Copies  of  the  frameworks  for  these  assessments  are  available  from  the  National  Assessment  Governing  Board  (NAGB) 
http://www.  nagb.  org. 

^ Not  all  participating  jurisdictions  gathered  both  public-  and  nonpublic-school  samples. 


new  trend  line.  The  assessment  involved  three  different  types  of  testing.  Some  portions 
covered  general  scientific  knowledge  and  skills.  Others  sections  of  the  survey  tested 
students  abilities  to  answer  questions  in  an  in-depth  thematic  or  topical  area.  Finally, 
each  sampled  student  completed  a component  that  involved  conducting  a hands-on 
science  experiment.  A combination  of  multiple-choice,  short  constructed-response,  and 
extended  constructed-response  questions  made  up  the  assessment;  in  aggregate,  well 
over  half  of  the  student  assessment  time  was  spent  answering  constructed-response 
questions.  In  addition  to  the  main  assessment,  a special  study  of  students  with  advanced 
scientific  training  was  conducted  at  grade  12. 

From  its  inception,  NAEP  has  developed  assessments  through  a consensus  process  and  the  1996 
instruments  were  no  exception.  Under  the  direction  of  the  National  Assessment  Governing  Board 
(NAGB),  educators,  scholars,  and  citizens  representative  of  many  diverse  constituencies  and  points  of 
view  designed  assessment  frameworks  for  both  subject  areas.  Staff  at  Educational  Testing  Service  (ETS) 
who  were  subject-area  experts  in  their  respective  fields  worked  with  subject-area  consultants  well  versed 
in  assessment  methodology  to  develop  assessment  questions  appropriate  to  the  objectives.  All  questions 
underwent  extensive  reviews  by  subject-matter  specialists,  measurement  specialists,  and  ETS  employees. 
Questions  were  assembled  and  printed  into  booklets  suitable  for  matrix  sampling  and  then  administered 
either  by  a trained  field  staff  (for  the  national  program)  or  by  state  or  local  school  district  staff  (for  the 
State  Assessment  program)  to  stratified,  multistage  probability  samples  of  students. 

All  1996  assessment  development  efforts  were  governed  by  four  major  considerations: 

1 . The  primary  goal  of  the  development  process  was  to  craft  instruments  that  matched 
the  content  definitions  included  in  the  assessment  frameworks,  which  was  developed 
through  consensus  processes  conducted  under  the  auspices  of  the  National 
Assessment  Governing  Board  (NAGB). 

2.  As  outlined  in  the  ETS  proposal  for  the  administration  of  the  NAEP  cooperative 
agreement  (ETS,  1992),  the  development  of  the  items  was  guided  by  an  Instrument 
Development  Committee  and  further  reviewed  by  state  representatives  and 
classroom  teachers  from  across  the  country.  In  addition,  the  items  had  to  be  carefully 
reviewed  for  potential  bias. 

3.  As  described  in  the  ETS  Standards  of  Quality  and  Fairness  (ETS,  1987),  all 
materials  developed  at  ETS  were  in  compliance  with  specified  procedures.  In 
particular,  all  questions  were  carefully  reviewed  for  content  accuracy, 
testworthiness,  and  potential  bias. 

4.  As  per  federal  regulations,  all  NAEP  cognitive  and  background  items  were  submitted 
to  a federal  clearance  process.  This  process  involved  review  of  all  cognitive  items  by 
NCES  and  NAGB,  and  review  of  all  background  questions  by  the  Office  of 
Management  and  Budget  (0MB),  the  Information  Management  Team  (IMT)  of  the 
Department  of  Education,  and  NCES. 

The  development  effort  for  the  1996  assessment  included  questionnaires''  for  students,  teachers, 
and  school  administrators,  in  addition  to  a substantial  number  of  cognitive  items  for  both  subject  areas. 
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These  questionnaires  can  be  obtained  from  the  National  Center  for 


Education  Statistics  (NCES). 
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The  following  sections  include  general  overviews  about  setting  objectives  and  developing  items 
and  specific  details  about  developing  subject-specific  objectives  and  assessments.  A list  of  the 
consultants  who  participated  in  the  1996  development  process  is  included  in  Appendix  A. 


2.2  GENERAL  OVERVIEW  OF  THE  1996  ASSESSMENT  FRAMEWORKS 

The  subject-area  objectives  for  each  NAEP  assessment  are  determined  through  a legislatively 
mandated  consensus  process.  These  objectives  typically  take  the  form  of  frameworks,  or  matrices, 
delineating  the  important  content  and  process  areas  to  be  assessed.  In  addition  to  these  broad  • 
frameworks,  the  Council  of  Chief  State  School  Officers  (CCSSO),  and  therefore  NAGB,  provided 
detailed  descriptions  of  item  types  and  the  numbers  of  items  to  be  selected  for  each  category.  The  various 
frameworks  for  the  1996  assessments  are  described  below. 

The  frameworks  for  the  main  1996  NAEP  assessments  were  developed  through  consensus 
processes  conducted  by  the  CCSSO  working  under  contract  to  NAGB.  The  projects  involved 
participation  and  review  by  many  groups,  including  teachers,  content-area  scholars,  educational 
policymakers,  and  members  of  the  general  public.  In  addition  to  people  directly  involved  in  the 
framework  development  processes,  the  documents  were  reviewed  by  state  education  and  testing  officials, 
by  representatives  of  professional  associations,  and  by  researchers.  In  addition,  the  frameworks  were  the 
subject  of  testimony  at  public  hearings  arranged  to  allow  the  widest  possible  participation  in  the 
consensus  process;  The  objectives  resulting  from  these  processes  reflect  neither  a narrowly  defined 
theoretical  framework  nor  the  view  of  every  participant.  They  do,  however,  represent  the  thinking  of  a 
broad  cross-section  of  individuals  who  are  expert  in  the  various  content  areas  and  who  are  deeply 
committed  to  the  improvement  of  American  education. 

The  framework  that  governs  the  1996  NAEP  mathematics  assessment  is  an  enhanced  version  of 
the  framework  that  governed  the  development  of  the  1990  and  1992  assessments.  This  framework  was 
originally  developed  under  the  auspices  of  the  Council  of  Chief  State  School  Officers  (CCSSO);  it  was 
revised  in  1991-1992  through  a consensus  project  managed  by  the  College  Board.  It  was  this  enhanced 
framework  that  governed  the  instrument  development  activities  related  to  the  1996  assessment.  The 
revisions  to  the  framework  were  minor,  and  allow  for  continued  measurement  of  educational  progress 
through  comparisons  with  1990  and  1992  results.  The  mathematics  framework  is  organized  according  to 
a five-by-three  matrix  of  content  strands  by  mathematical  abilities.  The  content  strands,  which  make  up 
the  main  reporting  subscales,  are: 

• Number  Sense,  Properties,  and  Operations; 

• Measurement; 

• Geometry  and  Spatial  Sense; 

• Data  Analysis,  Statistics,  and  Probability;  and 

• Algebra  and  Functions. 

In  addition,  the  assessment  was  designed  to  measure  the  three  mathematical  abilities  of: 

• Conceptual  Understanding, 

• Procedural  Knowledge,  and 

• Problem  Solving. 
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All  exercises  in  the  assessment  were  classified  as  measuring  one  of  the  content  strands  and  one 
of  the  abilities.  Additional  specifications  in  the  framework  are  related  to  an  assessment  dimension 
referred  to  as  mathematical  power.  Mathematical  power  is  conceived  as  consisting  of  mathematical 
abilities  within  a broader  context  of  reasoning  and  with  connections  across  the  broad  scope  of 
mathematical  content  and  thinking.  Communication  is  viewed  as  both  a unifying  thread  and  a way  for 
students  to  provide  meaningful  responses  to  tasks. 

The  framework  for  the  1996  science  assessment  is  structured  according  to  a matrix  organized 
according  to  two  major  dimensions:  fields  of  science  and  knowing  and  doing  science.  The  fields  of 
science,  which  make  up  the  subscales  on  which  assessment  results  are  analyzed  and  reported,  are  the 
earth,  physical,  and  life  sciences.  The  cognitive  dimension,  knowing  and  doing  science,  is  organized  into 
three  categories,  conceptual  understanding,  scientific  investigation,  and  practical  reasoning.  In  addition, 
the  framework  includes  requirements  for  measurement  of  content  that  crosses  other  categorical 
boundaries.  Specifically,  the  nature  of  science  and  themes  are  categories  that  should  integrate  the  three 
fields  of  science,  rather  than  represent  separate  content. 


2.3  GENERAL  OVERVIEW  OF  PROCEDURES  FOR  DEVELOPING  THE  ITEMS 

A carefully  developed  and  tested  series  of  steps,  similar  to  those  used  for  past  NAEP 
assessments,  was  utilized  to  create  assessment  items  that  reflected  mathematics  and  science  objectives 
and  that  measured  achievement  related  to  them  (see  Sections  2.4  through  2.6  for  more  detail).  The  steps 
were  as  follows: 

1 . NAGB  provided  item  specifications  and  frameworks  in  each  subject  area. 

2.  The  Instrument  Development  Committees  in  both  subject  areas  provided  guidance  to 
NAEP  staff  about  how  the  objectives  could  be  measured  given  the  realistic 
constraints  of  resources  and  the  feasibility  of  measurement  technology.  The 
committees  made  recommendations  about  priorities  for  the  assessment  (within  the 
context  of  the  assessment  framework)  and  the  types  of  items  to  be  developed. 

3.  Items  were  chosen  for  the  assessment  through  an  extensive  selection  process  that 
involved  the  input  of  practitioners  from  across  the  country  as  well  as  from  members 
of  the  Instrument  Development  Committees. 

4.  Specialists  with  subject-matter  expertise,  skills,  and  experience  in  creating  items 
according  to  specifications  were  identified  from  inside  and  outside  ETS  to  develop 
and  review  the  assessment  questions. 

5.  The  items  were  reviewed  and  revised  by  NAEP/ETS  staff  and  external  test 
specialists. 

6.  Representatives  from  the  State  Education  Agencies  met  and  reviewed  all  items  and 
background  questionnaires  that  were  scheduled  to  be  part  of  the  state  assessment. 

7.  Language  editing  and  sensitivity  reviews  were  conducted  as  required  by  the  ETS 
Standards  for  Quality  and  Fairness. 

8.  Field  test  materials  were  prepared,  including  those  necessary  to  secure  clearance  by 
the  Office  of  Management  and  Budget. 
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9.  A field  test  was  conducted  in  many  states,  the  District  of  Columbia,  and  three 
territories.^ 

10.  Representatives  from  State  Education  Agencies  met  and  reviewed  the  field  test 
results  for  all  exercises  selected  for  the  state  assessment. 

11.  Based  on  the  field  test  analyses,  new  items  for  the  1996  assessment  were  revised, 
modified,  and  re-edited,  where  necessary.  The  items  once  again  underwent  the  full 
range  of  ETS  reviews. 

12.  The  Instrument  Development  Committees  approved  the  selection  of  items  to  include 
in  the  1996  assessment. 

13.  After  a final  review  and  check  to  ensure  that  each  assessment  booklet  and  each  block 
met  the  overall  guidelines  for  the  assessment,  the  booklets  were  typeset  and  printed. 

The  following  sections  describe  the  development  of  the  mathematics  and  science  assessments  in 
more  detail. 


2.4  DEVELOPING  THE  MATHEMATICS  ASSESSMENT 
2.4.1  Overview 

The  framework  that  governs  the  1996  NAEP  mathematics  assessment  is  an  enhanced  version  of 
the  framework  used  on  the  1990  and  1992  assessment.  Similar  to  other  NAEP  assessments,  the  1990 
mathematics  framework  was  developed  through  a broad-based  consensus  process  managed  by  the 
CCSSO.  In  1991-1992,  the  National  Assessment  Governing  Board  (NAGB)  contracted  with  The  College 
Board  to  review  and  revise  the  framework  in  preparation  for  the  assessment  originally  planned  for  1994 
and  administered,  in  fact,  in  1996.  The  development  process  involved  a committee  of  mathematicians  and 
mathematics  education  specialists.  Educators,  scholars,  and  citizens,  representative  of  many  diverse 
constituencies  and  points  of  view,  participated  in  the  national  consensus  process  to  review  and  revise 
objectives  for  the  assessment. 

The  instrument  used  in  the  1996  mathematics  assessment  was  composed  of  a combination  of  new 
items  developed  for  administration  ih  1996^  and  items  from  the  1992  and  1990  assessments.  Those  items 
that  were  carried  over  from  the  1992  and  1990  instruments  comprised  approximately  60  percent  of  the 
1996  instrument.  The  remaining  portion  was  made  up  of  new  items  developed  according  to  the 
recommendations  included  in  the  enhanced  assessment  specifications.  Maintaining  approximately  60 
percent  of  the  instrument  across  the  two  assessment  years  (1992  and  1996)  allowed  for  the  reporting  of 


* In  this  case,  two  field  tests  were  conducted.  The  mathematics  and  science  assessments  were  originally  scheduled  for  1994;  thus 
a 1993  field  test  was  conducted.  This  field  test  was  designed  to  provide  replacement  items  for  those  released  in  mathematics  and 
all  exercises  needed  in  the  new  science  assessment.  Because  the  assessments  were  delayed  until  1996,  a supplementary  field  test 
was  conducted  in  1995.  In  mathematics,  this  field  test  was  used  for  the  development  of  theme  and  advanced  blocks,  and  in  the 
development  of  assessment  accommodations.  In  science,  the  1995  field  test  was  used  to  develop  more  general  science  exercises 
for  the  main  assessment. 

* As  was  noted  above,  many  of  the  new  items  were  originally  scheduled  for  use  in  1994;  however,  the  mathematics  assessment 
was  deferred'until  1996.  Other  items  were  developed  and  field  tested  in  1995.  For  purposes  of  this  report,  we  will  refer  to  all 
exercises  that  were  used  in  1996  but  that  were  not  part  of  earlier  NAEP  surveys  as  having  been  “newly  developed  for  1996.” 
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trends  in  mathematics  performance.  At  the  same  time,  developing  a new  set  of  items  made  it  possible  to 
release  approximately  40  percent  of  the  1992  assessment  for  public  use. 

In  developing  the  new  portion  of  the  1996  NAEP  mathematics  assessment,  the  same  procedures 
used  in  1992  were  followed;  however,  new  items  were  constructed  to  meet  the  demands  of  the  revised 
framework.  All  items  underwent  extensive  reviews  by  specialists  in  mathematics,  measurement,  and 
bias/sensitivity;  items  developed  for  grades  four  and  eight  were  also  reviewed  by  state  representatives. 
The  core  goals  of  the  ETS  assessment  development  process  and  procedures  used  to  realize  these  goals 
are  outlined  in  the  introduction  to  this  chapter  and  in  Section  2.3. 

The  following  sections  include  a detailed  description  of  the  development  of  the  framework, 
objectives,  and  items  for  the  1996  NAEP  mathematics  assessment.  Section  2.4.8  describes  the  student 
background  questionnaires  and  the  reading  teacher  questionnaire.  Additional  information  on  the  structure 
and  content  of  assessment  booklets  can  be  found  in  Chapter  4.  Various  committees  worked  on  the 
development  of  the  framework,  objectives,  and  items  for  the  mathematics  assessment.  The  list  of 
committee  members  and  consultants  who  participated  in  the  1996  development  process  is  provided  in 
Appendix  A. 


2.4.2  Development  of  the  Assessment  Framework 

NAGB  is  responsible  for  developing  assessment  objectives  and  test  specifications  for  NAEP 
surveys.  Appointed  by  the  Secretary  of  Education  from  lists  of  nominees  proposed  by  the  board  itself  in 
various  statutory  categories,  the  26-member  board  is  composed  of  state,  local,  and  federal  officials,  as 
well  as  educators  and  members  of  the  public. 

Under  contract  with  NAGB,  The  College  Board  convened  a committee  during  1991  and  1992  to 
develop  an  enhanced  version  of  the  framework  that  had  been  used  during  the  development  of  the  1990 
and  1992  assessments  (the  1992  mathematics  assessment  had  already  been  developed  at  the  time  the 
development  of  this  enhanced  framework  was  begun).  The  enhanced  version  was  needed  to  better  reflect 
the  rapid  evolution  of  mathematics  instruction  that  was  underway  in  the  early  1990s  as  a result  of  the 
emergence  of  the  National  Council  of  Teachers  of  Mathematics  (NCTM)  Curriculum  and  Evaluation 
Standards  for  School  Mathematics.  The  Standards  were  rapidly  becoming  one  of  the  acknowledged 
barometers  for  measuring  achievement.  The  development  process  for  the  enhanced  framework  was  based 
on  consensus  building,  and  included  the  committee  listed  in  Appendix  A. 

During  this  development  process,  input  and  reactions  were  continually  sought  from  a wide  range 
of  members  of  the  mathematics  field,  experts  in  assessment,  school  administrators,  and  state  staff.  In 
particular,  the  process  was  informed  by  recommendations  of  leading  professional  organizations  in 
mathematics. 


2.4.3  Framework  and  Assessment  Design  Principles 

The  planning  committee  involved  in  the  review  and  revision  of  the  NAEP  mathematics 
framework  was  given  a number  of  working  guidelines  and  goals  by  the  National  Assessment  Governing 
Board  and  by  the  steering  committee  that  oversaw  the  process.  These  guidelines  directed  the  planning 
committee  to  develop  assessment  specifications  that  called  for  performance-oriented  exercises  that  focus 
on  problem-solving  and  provide  students  with  opportunities  to  communicate  their  understandings  in 
mathematics.  The  framework  should,  according  to  these  guidelines,  embody  a broad  view  of  mathematics 


that  addressed  the  high  levels  of  mathematical  literacy  needed  for  employability,  personal  development, 
and  citizenship.  Also,  the  framework  should  take  into  account  findings  of  contemporary  research  on 
mathematics  and  mathematics  education,  and  would  expand  the  range  of  assessment  tools  to  include 
formats  that  more  closely  resembled  classroom  activities. 

The  development  was  further  guided  by  the  consideration  that  the  assessment  should  reflect 
many  of  the  states’  curricular  emphases  and  objectives  in  addition  to  what  various  scholars,  practitioners, 
and  interested  citizens  believed  should  be  included  in  the  curriculum.  Accordingly,  the  committee 
focused  on  several  general  principles  in  revising  the  NAEP  mathematics  assessment.  These  principles 
are: 

• The  purpose  of  the  NAEP  mathematics  assessment  is  to  provide  information  about 
the  progress  and  achievement  of  students  in  general  rather  than  to  test  individual 
students’  ability.  NAEP  is  designed  to  inform  policymakers  and  the  public  about 
mathematics  ability  in  the  United  States.  Furthermore,  NAEP  state  data  can  be  used 
to  inform  states  of  their  students’  relative  strengths  and  weaknesses. 

• The  term  “mathematical  literacy”  encompasses  such  broad  skills  and  abilities  as 
being  able  to  reason  numerically,  algebraically,  geometrically,  spatially,  and  with 
data;  identify  and  apply  problem-solving  strategies  appropriately  in  situations;  and 
use  the  language  of  mathematics  to  construct  clear  and  coherent  responses  to 
problems  or  tasks. 

• The  mathematics  assessment  should  use  authentic  problems  and  tasks  that  address 
important  mathematics  concepts  and  skills  so  that  the  assessment  tool  will 
demonstrate  a close  link  to  desired  classroom  instruction  and  students’  mathematics 
experiences. 

• Every  effort  should  be  made  to  make  the  best  use  of  available  methodology  and 
resources  in  driving  assessment  capabilities  forward. 

• Every  effort  must  be  made  in  developing  the  assessment  to  represent  a variety  of 
opinions,  perspectives,  and  emphases  among  professionals  in  universities,  as  well  as 
in  state  and  local  school  districts. 


2.4.4  Framework  for  the  1996  Assessment 

The  framework  for  the  1996  mathematics  assessment  is  organized  according  to  a five-by-three 
matrix  of  content  strands  by  mathematical  abilities.  The  content  strands  are: 

• Number  Sense,  Properties,  and  Operations; 

• Measurement; 

• Geometry  and  Spatial  Sense; 

• Data  Analysis,  Statistics,  and  Probability;  and 

• Algebra  and  Functions. 
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These  content  strands  were  assessed  across  the  three  mathematical  abilities  of: 


• Conceptual  Understanding, 

• Procedural  Knowledge,  and 

• Problem  Solving. 

Figures  2-1  and  2-2  describe  the  five  content  strands  and  three  mathematical  abilities  that  guided 
the  development  of  the  1996  mathematics  assessment. 


Figure  2-1 

Descriptions  of  Content  Strands  in  Mathematics 
Number  Sense,  Properties,  and  Operations 

This  strand  focuses  on  students’  understanding  of  numbers  (whole  numbers,  fractions,  decimals, 
integers,  real  numbers,  and  complex  numbers),  operations,  and  estimation,  and  their  application  to  real- 
world  situations.  Students  will  be  expected  to  demonstrate  an  understanding  of  numerical  relationships  as 
expressed  in  ratios,  proportions,  and  percents.  Students  also  will  be  expected  to  understand  properties  of 
numbers  and  operations,  generalize  from  number  patterns,  and  verify  results. 

Measurement 

The  measurement  strand  focuses  on  understanding  of  the  process  of  measurement  and  on  the  use 
of  numbers  and  measures  to  describe  and  compare  mathematical  and  real-world  objects.  Students  will  be 
asked  to  identify  attributes,  select  appropriate  units  and  tools,  apply  measurement  concepts,  and 
communicate  measurement-related  ideas. 


Geometry  and  Spatial  Sense 

As  described  in  the  NCTM  Standards,  spatial  sense  must  be  an  integral  component  of  the  study 
and  assessment  of  geometry.  Understanding  spatial  relationships  allows  students  to  use  the  dynamic 
nature  of  geometry  to  connect  mathematics  to  their  world. 

This  content  strand  is  designed  to  extend  well  beyond  low-level  identification  of  geometric 
shapes  into  transformations  and  combinations  of  those  shapes.  Informal  constructions  and 
demonstrations  (including  drawing  representations),  along  with  their  justifications,  take  precedence  over 
more  traditional  types  of  compass-and-straightedge  constructions  and  proofs.  While  reasoning  is 
addressed  throughout  all  of  the  content  strands,  this  strand  continues  to  lend  itself  to  the  demonstration  of 
reasoning  within  both  formal  and  informal  settings.  The  extension  of  proportional  thinking  to  similar 
figures  and  indirect  measurement  is  an  important  connection  here. 

(continued) 
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Figure  2-1  (continued) 

Descriptions  of  Content  Strands  in  Mathematics 
Data  Analysis,  Statistics,  and  Probability 

The  important  skills  of  collecting,  organizing,  reading,  representing,  and  interpreting  data  will  be 
assessed  in  a variety  of  contexts  to  reflect  the  pervasive  use  of  these  skills  in  dealing  with  information. 
Statistics  and  statistical  concepts  extend  these  basic  skills  to  include  analyzing  and  communicating 
increasingly  sophisticated  interpretations  of  data.  Dealing  with  uncertainty  and  making  predictions  about 
outcomes  require  an  understanding  not  only  of  the  meaning  of  basic  probability  concepts  but  also  the 
application  of  those  concepts  in  problem-solving  and  decision-making  situations. 

Questions  will  emphasize  appropriate  methods  for  gathering  data,  the  visual  exploration  of  data, 
a variety  of  ways  of  representing  data,  and  the  development  and  evaluation  of  arguments  based  on  data 
analysis.  Students  will  be  expected  to  apply  these  ideas  in  increasingly  sophisticated  situations  that 
require  increasingly  comprehensive  analysis  and  decision  making. 

Algebra  and  Functions 

This  strand  extends  from  work  with  simple  patterns  at  grade  4,  to  basic  algebra  concepts  at  grade 
8,  to  sophisticated  analysis  at  grade  12,  and  involves  not  only  algebra  but  also  precalculus  and  some 
topics  from  discrete  mathematics.  As  described  in  the  NCTM  Standards,  these  algebraic  concepts  are 
developed  throughout  the  grades  with  informal  modeling  done  at  the  elementary  level  and  with  increased 
emphasis  on  functions  at  the  secondary  level.  The  nature  of  the  algebraic  concepts  and  procedures 
included  in  the  assessment  at  all  levels  reflects  the  NCTM  Standards.  Students  will  be  expected  to  use 
algebraic  notation  and  thinking  in  meaningful  contexts  to  solve  mathematical  and  real-world  problems, 
specifically  addressing  an  increasing  understanding  of  the  use  of  functions  (including  algebraic  and 
geometric)  as  a representational  tool. 
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Figure  2-2 

Descriptions  of  Mathematical  Abilities 

Conceptual  Understanding 

Students  demonstrate  conceptual  understanding  in  mathematics  when  they  provide  evidence  that 
they  can  recognize,  label,  and  generate  examples  and  nonexamples  of  concepts;  use  and  interrelate 
models,  diagrams,  manipulatives,  and  varied  representations  of  concepts;  identify  and  apply  principles 
(i.e.,  valid  statements  generalizing  relationships  among  concepts  in  conditional  form);  know  and  apply 
facts  and  definitions;  compare,  contrast,  and  integrate  related  concepts  and  principles  to  extend  the  nature 
of  concepts  and  principles;  recognize,  interpret,  and  apply  the  signs,  symbols,  and  terms  used  to  represent 
concepts;  or  interpret  the  assumptions  and  relations  involving  concepts  in  mathematical  settings. 

Conceptual  understanding  reflects  a student’s  ability  to  reason  in  settings  involving  the  careful 
application  of  concept  definitions,  relations,  or  representations  of  either.  Such  an  ability  is  reflected  by 
student  performance  that  indicates  the  production  of  examples,  common  or  unique  representations,  or 
communications  indicating  the  ability  to  manipulate  central  ideas  about  the  understanding  of  a concept  in* 
a variety  of  ways. 

Procedural  Knowledge 

Students  demonstrate  procedural  knowledge  in  mathematics  when  they  select  and  apply 
appropriate  procedures  correctly;  verify  or  justify  the  correctness  of  a procedure  using  concrete  models 
or  symbolic  methods;  or  extend  or  modify  procedures  to  deal  with  factors  inherent  in  problem  settings. 

Procedural  knowledge  includes  the  various  numerical  algorithms  in  mathematics  that  have  been 
created  as  tools  to  meet  specific  needs  efficiently.  Procedural  knowledge  also  encompasses  the  abilities 
to  read  and  produce  graphs  and  tables,  execute  geometric  constructions,  and  perform  noncomputational 
skills  such  as  rounding  and  ordering.  These  latter  activities  can  be  differentiated  from  conceptual 
understanding  by  the  task  context  or  presumed  student  background — that  is,  an  assumption  that  the 
student  has  the  conceptual  understanding  of  a representation  and  can  apply  it  as  a tool  to  create  a product 
or  to  achieve  a numerical  result.  In  these  settings,  the  assessment  question  is  how  well  the  student 
executed  a procedure  or  how  well  the  student  selected  the  appropriate  procedure  to  effect  a given  task. 

Procedural  knowledge  is  often  reflected  in  a student’s  ability  to  connect  an  algorithmic  process 
with  a given  problem  situation,  to  employ  that  algorithm  correctly,  and  to  communicate  the  results  of  the 
algorithm  in  the  context  of  the  problem  setting.  Procedural  understanding  also  encompasses  a student’s 
ability  to  reason  through  a situation,  describing  why  a particular  procedure  will  give  the  correct  answer 
for  a problem  in  the  context  described. 

Problem  Solving 

In  problem  solving,  students  are  required  to  use  their  accumulated  knowledge  of  mathematics  in 
new  situations.  Problem  solving  requires  students  to  recognize  and  formulate  problems;  determine  the 
sufficiency  and  consistency  of  data;  use  strategies,  data,  models,  and  relevant  mathematics;  generate, 
extend,  and  modify  procedures;  use  reasoning  (i.e.,  spatial,  inductive,  deductive,  statistical,  or 
proportional)  in  new  settings;  and  judge  the  reasonableness  and  correctness  of  solutions.  Problem  solving 
situations  require  students  to  connect  all  of  their  mathematical  knowledge  of  concepts,  procedures, 
reasoning,  and  communication/  representational  skills  in  confronting  new  situations.  As  such,  these 
situations  are,  perhaps,  the  most  accurate  measures  of  students’  proficiency  in  mathematics. 


Tables  2-1  and  2-2  show  the  percentages  of  assessment  time  that  the  framework  indicates  should 
be  devoted  to  each  content  strand  and  mathematical  ability. 


Table  2-1 

Percentage  Distribution  of  Items  by  Grade  and  Content  Strand 
as  Specified  in  the  NAEP  Mathematics  Framework 


Content  Strand 

Grade  4 

Grade  8 

Grade  12 

Number  Sense,  Properties,  and  Operations 

' 40%-70% 

25%-60% 

20%-50% 

Measurement 

20% 

15% 

15% 

Geometry  and  Spatial  Sense 

15% 

20% 

20% 

Data  Analysis,  Statistics,  and  Probability 

10% 

15% 

20% 

Algebra  and  Functions 

15% 

25% 

25% 

^ For  this  category,  these  percentages  are  the  minimum  and  maximum  that  are  acceptable,  respectively. 

Table  2-2 

Percentage  Distribution  of  Items  by  Grade  and  Mathematical  Ability 

as  Specified  in 

the  NAEP  Mathematics  Framework 

Mathematical  Ability  Grade  4 

Grade  8 

Grade  12 

Conceptual  Understanding 

33% 

33% 

33% 

Procedural  Knowledge 

33% 

33% 

33% 

Problem  Solving 

33% 

33% 

33% 

Note:  Some  items  carry  multiple  classifications. 


2.4.5  Developing  the  Cognitive  Items 

The  1996  assessment  was  designed  to  serve  a dual  purpose:  to  meet  the  content  specifications 
elaborated  in  the  revised  NAEP  mathematics  framework  and  to  allow  for  the  measurement  of  changes  in 
student  mathematics  achievement  (that  is,  to  permit  linking  to  the  1990  and  1992  NAEP  mathematics 
assessment).  Because  of  these  objectives,  the  following  strategies  were  adopted  in  developing  the  1996 
mathematics  assessment.  First,  at  each  grade,  8 of  the  13  blocks  used  in  the  1992  assessment  were  carried 
forward  and  used  in  1996.  Items  were  developed  and  field  tested  in  1993  to  replace  the  five  blocks  from 
the  1992  assessment  that  had  been  released  for  public  use.  In  addition,  other  portions  of  the  1993  field 
test  were  used  to  develop  special  components  of  the  assessment.  To  this  end,  a new  block  of  “estimation” 
exercises  was  developed,  as  were  exercises  for  a special  study  of  students  at  grades  eight  and  twelve  who 
had  received  advanced  mathematics  training.  Given  the  fact  that  the  mathematics  assessment  was  delayed 
from  1994  until  1996,  a supplemental  field  test  was  held  in  1995  (exercises  were  developed  for  this  field 
test  in  1994).  This  round  of  exercise  development  was  used  to  build  extra  exercises  for  the  estimation  and 
advanced  components  of  the  assessment.  It  was  also  used  to  build  “theme  blocks,”  which  were  intended 
to  measure  in-depth  mathematics  problem-solving  ability. 

The  development  of  cognitive  items  involved  careful  field  testing,  both  locally  and  nationally,  of 
grade-appropriate  questions  and  tasks  for  the  assessment.  Items  were  selected  from  a pool  of  questions 
that  were  written  by  teachers  from  across  the  country  as  well  as  by  mathematics  assessment  specialists  on 
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staff  at  ETS.  The  framework  stated  that  the  assessment  should  include  some  performance-based 
questions  and  tasks  that  require  students  to  reason  and  make  connections  within  and  across  different 
content  strands  of  mathematics.  Final  selections  of  questions  used  in  the  1996  assessment  were  approved 
by  the  Mathematics  Instrument  Development  Committee. 

The  assessment  included  constructed-response  (short  and  extended)  and  multiple-choice  items. 
The  decision  to  use  a specific  item  type  was  based  on  a consideration  of  the  most  appropriate  format  for 
assessing  the  particular  objective.  Both  types  of  constructed-response  items  were  designed  to  provide  an 
in-depth  view  of  students’  ability  to  communicate  their  understanding  of  important  concepts  in 
mathematics.  Short  constructed-response  questions  (scored  with  either  a 2-  or  3-level  scoring  rubric) 
were  used  when  students  needed  to  respond  briefly  in  order  to  demonstrate  full  comprehension.  Extended 
constructed-response  questions  (scored  with  a 4-  or  5-level  scoring  rubric)  were  used  when  the  task 
required  more  thoughtful  consideration  of  the  problem  and  engagement  in  more  complex  reasoning 
processes.  Multiple-choice  items  were  used  when  a straightforward,  single  correct  answer  was  required. 

A carefully  developed  and  proven  series  of  steps  was  used  to  create  the  assessment  items.  These 
steps  are  described  earlier  in  the  chapter  under  Section  2.3. 

As  was  mentioned  above,  the  assessment  was  designed  to  allow  for  measurement  of  trends. 
Therefore,  eight  15-minute  blocks  at  each  grade  were  included  in  the  1996  assessment  that  had  been  a 
part  of  the  1992  assessment  (and  in  some  cases  the  1990  assessment).  These  blocks  were  used  in 
precisely  the  same  form  as  they  were  in  1992.  In  addition,  one  of  the  paced-tape  estimation  blocks  was 
carried  forward  from  1992.  The  remainder  of  the  exercises  used  in  1996  were  newly  developed  during 
either  the  1992-1993  or  1994-1995  development  cycles. 

2.4.6  Development  of  the  Operational  Forms 

The  field  tests  of  new  items  for  the  1996  assessment  were  conducted  in  February  and  March, 

1993  and  February  and  March,  1995.  The  field  test  involved  a convenience  sample  in  which  roughly  500 
responses  were  obtained  to  each  item. 

The  field  test  data  were  collected,  scored,  and  analyzed  in  preparation  for  meetings  with  the 
Mathematics  Instrument  Development  Panel.  The  objectives  that  guided  the  review  of  these  items  were: 

• to  determine  which  items  were  most  suitable  for  assessing  understanding  in 
mathematics  in  accordance  with  the  framework; 

• to  select  items  that  displayed  appropriate  statistical  attributes; 

• to  determine  the  need  for  revisions  of  items  that  lacked  clarity,  or  had  ineffective 
item  formats; 

• to  prioritize  items  to  be  included  in  the  assessment;  and, 

• to  determine  appropriate  timing  for  assessment  items. 

Committee  members,  ETS  assessment  staff,  and  NAEP/ETS  staff  reviewed  the  materials.  Item 
analyses  (which  provided  the  mean  percentage  of  correct  responses,  the  r-biserial  correlations,  and  the 
difficulty  level  for  each  item)  were  used  as  a guide  in  identifying  and  flagging  for  further  review  those 


test  questions  that  were  not  measuring  the  intended  objective  well;  In  addition,  another  meeting  of 
representatives  from  state  education  agencies  was  convened  to  review  the  field  test  results  for  exercises 
included  in  the  grade  four  or  eight  assessments. 

Once  the  committees  had  selected  the  items,  all  items  were  rechecked  for  content,  measurement, 
and  sensitivity  concerns.  The  federal  clearance  process  was  initiated  in  June  1993  with  the  submission  of 
draft  materials  to  NCES.  The  package  containing  the  set  of  cognitive  items  assembled  into  blocks  and 
questionnaires  was  submitted  in  August  1993.  A revised  package  with  the  new  thematic  blocks  and  the 
adjusted  advanced  and  estimation  blocks  was  submitted  in  July,  1995.  Throughout  the  clearance  process, 
revisions  were  made  in  accordance  with  changes  required  by  the  government.  After  approval,  the  blocks 
(assembled  into  booklets)  tmd  questionnaires  were  readied  for  printing  in  preparation  for  the  assessment. 

2.4.7  Distribution  of  Assessment  Items 

The  mathematics  assessment  developed  for  use  in  1996  was  organized  according  to  a series  of 
blocks,  each  containing  a set  of  questions.  Some  of  the  blocks  were  unique  to  a particular  grade  level. 
Other  blocks  were  designed  to  be  given  to  students  in  two  grades  (either  4 and  8 or  8 and  12)  and  a few 
blocks  (a  small  percentage  of  the  blocks  used  to  measure  trend)  overlap  all  three  grades. 

At  each  grade,  the  main  component  of  the  assessment  used  for  the  creation  of  reporting  scales 
included  thirteen  different  15-minute  blocks  of  multiple-choice  and  constructed-response  questions 
(some  with  both  regular  and  extended  constructed-response  questions).  Two  additional  books  containing 
30-minute  theme  blocks  of  multiple-choice  tmd  constructed-response  questions  were  also  used.  Two 
estimation  blocks  at  each  grade  were  part  of  a special  study  and  were  presented  to  students  by  a paced- 
audiotape  to  assess  students’ estimation  skills.  Two  20-minute  grade  8 special  study  blocks  and  two  30- 
minute  grade  12  special  study  blocks  targeted  students  in  advanced  mathematics  courses.  Of  the  13 
blocks  at  each  grade  that  were  used  for  the  main  assessment: 

• three  to  five  blocks  at  each  grade  level  included  items  designed  to  be  answered  using 
a calculator.  For  the  grade  4 calculator  blocks,  students  were  provided  with  a 
4-function  calculator,  while  at  grades  8 and  12  the  students  were  provided  with  a 
scientific  calculator  (students  were  also  provided  calculators  for  the  theme  blocks 
and  allowed  to  use  their  own  calculators  on  the  advanced  blocks) 

• one  block  at  all  grade  levels  contained  questions  requiring  the  use  of  a 
protractor/ruler  (ruler  only  for  grade  4) 

• two  blocks  at  each  grade  involved  the  use  of  manipulatives;  several  of  the  theme 
blocks  involved  the  use  of  manipulatives  as  well 

• seven  blocks  at  grade  4 and  eight  blocks  at  each  of  grades  8 and  12  included 
extended  constructed-response  items.  The  extended  constructed-response 
mathematics  items  call  for  the  student  to  work  through  a complex  problem,  require 
about  five  minutes  to  complete,  and  were  scored  on  a 0-5  point  scale.  The  theme  and 
advanced  blocks  also  included  extended  constructed-response  items. 
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The  sections  that  follow  discuss  the  distribution  of  exercises  on  the  mathematics  assessment.  For 
purposes  of  this  discussion,  we  will  limit  the  calculations  to  exercises  that  appeared  as  part  of  the  main 
assessment,  or  BIB.  Special  study,  theme,  and  estimation  blocks  will  be  discussed  below. 

Figure  2-3  lists  the  total  number  of  items  at  each  grade  level  in  the  main  portion  of  the  1996 
assessment.  Of  the  total  of  360  items,  there  were  194  unique  multiple-choice  items  and  166  unique 
constructed-response  items.  Some  of  these  items  were  used  at  more  than  one  grade  level.  As  a result,  the 
sum  of  the  items  that  appear  at  each  grade  level  is  greater  than  the  total  number  of  unique  items. 


Figure  2-3 

Total  Number  of  Items  for  the  1996  Mathematics  Main  Assessment 


Grade  4 

Grade  8 

Grade  12 

79  Multiple-Choice* 

92  Multiple-Choice* 

93  Multiple-Choice* 

58  Short  Constnicted-Response^ 

65  Short  Constnicted-Response^ 

66  Short  Constnicted-Response^ 

7 Extended  Constnicted-Response 

8 Extended  Constructed-Response 

8 Extended  Constnicted-Response 

\y 


\y 


V 


V 


50  Multiple-Choice 
33  Short  Constnicted-Response 
7 Extended  Constnicted-Response 


37  Multiple-Choice 
26  Short  Constnicted-Response 
8 Extended  Constnicted-Response 


52  Multiple-Choice 
45  Short  Constnicted-Response 
8 Extended  Constnicted-Response 


V 


14  Multiple-Choice 
1 8 Short  Constnicted-Response 
0 Extended  Constnicted-Response 


26  Multiple-Choice 
14  Short  Constnicted-Response 
0 Extended  Constnicted-Response 


^ These  item  counts  include  15  multiple-choice  questions  that  were  presented  at  all  three  grades. 

^ These  item  counts  include  7 short  constructed-response  questions  that  were  presented  at  all  three  grades. 


In  the  development  process,  every  effort  was  made  to  meet  the  content  and  process  targets 
specified  in  the  assessment  framework.  Tables  2-3,  2-4,  and  2-5  show  the  approximate  percentage  of 
aggregate  assessment  items  devoted  to  each  content  strand  at  each  grade  level.  Percentages  are  based  on 
the  classifications  agreed  upon  by  NAEP’s  1996  Mathematics  Instrument  Development  Committee,  and 
confirmed  by  independent  reviewers. 
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Table  2-3 

Distribution  of  Assessment  Items 
for  the  Mathematics  Assessment,  Grade  4 


1990  1992  1996 

Number  Percentage  Number  Percentage  Number  Percentage 


of  items 

Of  items 

of  items 

of  items 

of  items 

of  items 

Content  Strand 

Number  Sense,  Properties,  and  Operations 

52 

48% 

63 

40% 

59 

41% 

Measurement 

21 

19% 

31 

20% 

25 

17% 

Geometry  and  Spatial  Sense 

14 

13% 

27 

17% 

25 

17% 

Data  Analysis,  Statistics,  and  Probability 

8 

7% 

20 

12% 

17 

12% 

Algebra  and  Functions 

14 

13% 

17 

11% 

18 

13% 

Total 

109 

100% 

158 

100% 

144 

100% 

Mathematical  Ability 

Conceptual  Understanding 

42 

39% 

64 

40% 

61 

42% 

Procedural  Knowledge 

31 

28% 

31 

20% 

32 

22% 

Problem  Solving 

36 

33% 

63  . 

40% 

51 

35% 

Total 

109 

100% 

158 

100% 

144 

99% 

Table  2-4 

Distribution  of  Assessment  Items 
for  the  Mathematics  Assessment,  Grade  8 

" 1990  1992  1996 

Number  Percentage  Number  Percentage  Number  Percentage 


of  items 

Of  items 

of  items 

of  items 

of  items 

of  items 

Content  Strand 

Number  Sense,  Properties,  and  Operations 

45 

33% 

58 

32% 

48 

29% 

Measurement 

21 

15% 

32 

17% 

27 

16% 

Geometry  and  Spatial  Sense 

26 

19% 

36 

20% 

31 

19% 

Data  Analysis,  Statistics,  and  Probability 

19 

14% 

28 

15% 

25 

15% 

Algebra  and  Functions 

26 

19% 

29 

16% 

34 

21% 

Total 

137 

100% 

183 

100% 

165 

100% 

Mathematical  Ability 

Conceptual  Understanding 

59 

43% 

67 

37% 

57 

35% 

Procedural  Knowledge 

41 

30% 

45 

24% 

46 

28% 

Problem  Solving 

37 

27% 

71 

39% 

62 

38% 

Total 

137 

100% 

183 

100% 

165 

101% 

66 

o 
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Table  2-5 

Distribution  of  Assessment  Items 
for  the  Mathematics  Assessment,  Grade  12 


1990  1992  1996 


Number  Percentage  Number  Percentage  Number  Percentage 
of  items  Of  items  of  items  of  items  of  items  of  items 


Content  Strand 

Number  Sense,  Properties,  and  Operations 

37 

26% 

43 

24% 

41 

25% 

Measurement 

23 

16% 

29 

16% 

23 

14% 

Geometry  and  Spatial  Sense 

24 

17% 

31 

l'8% 

27 

16% 

Data  Analysis,  Statistics,  and  Probability 

22 

15% 

29 

16% 

34 

20% 

Algebra  and  Functions 

38 

26% 

47 

26% 

42 

25% 

Total 

144 

100% 

179 

100% 

167 

101% 

Mathematical  Ability 

Conceptual  Understanding 

53 

37% 

70 

39% 

66 

40% 

Procedural  Knowledge 

48 

33% 

52 

29% 

51 

31% 

Problem  Solving 

43 

30% 

57 

32% 

50 

30% 

Total 

144 

100% 

179 

100% 

167 

101% 

Before  proceeding,  it  is  worth  saying  a few  words  about  the  other  assessment  components.  The 
estimation  blocks  were  given  to  nationally  representative  samples  of  students  at  each  grade.  Estimation 
was  not  given  at  the  state  level.  Students  in  the  estimation  sample  took  one  block  from  the  main  BIB.  In 
addition,  they  took  two  estimation  blocks.  Both  estimation  blocks  were  administered  by  a paced-tape 
method.^  The  first  estimation  block  was  all  multiple-choice,  and  was  carried  forward  intact  from  1992.  It 
contained  20  items  at  grade  4,  and  22  items  at  grades  8 and  12.  The  second  estimation  block  was  newly 
developed  for  1996.  It  contained  13  items  at  grade  4 (10  multiple  choice  and  3 constructed-response),  15 
at  grade  8 (9  multiple-choice),  and  16  at  grade  12  (all  multiple-choice).  Trend  was  measured  on  this 
assessment  component. 

Theme  blocks  were  also  given  to  nationally-representative  samples  at  all  grades.  Students  in  the 
theme  sample  first  completed  one  block  from  the  main  assessment  BIB.  Then  they  were  administered  one 
30-minute  theme  block.  These  blocks  called  for  in-depth  examination  of  mathematical  problems,  and  for 
the  use  of  a variety  of  mathematics  skills.  At  grade  4,  the  two  theme  blocks  included  a total  of  one 
multiple-choice  question  and  13  constructed-response  items.  At  grade  8,  the  total  was  8 multiple-choice 
items  and  13  constructed-response  items.  At  grade  12,  the  total  was  8 multiple-choice  items  and  10 
constructed-response  exercises. 

Finally,  the  advanced  special  studies  were  administered  to  samples  of  students  meeting  certain 
course-taking  criteria  at  grades  8 and  12  (algebra  or  higher  at  grade  8,  and  calculus,  precalculus,  or 
algebra  three  at  grade  12).  Students  took  a linking  block  composed  of  exercises  from  the  main  assessment 
and  then  two  20-minute  blocks  (at  grade  8)  or  two  30-minute  blocks  (grade  12)  of  advanced  exercises. 
These  blocks  contained  9 multiple-choice  and  13  constructed-response  items  at  grade  8,  and  7 multiple- 
choice  and  15  constructed-response  items  at  grade  12. 


^ The  blocks  were  paced  so  students  had  to  estimate  rather  than  calculate  the  answer. 
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2.4.8  Questionnaires 


. As  part  of  the  national  assessment  (as  well  as  the  State  Assessment),  a series  of  questionnaires 
was  administered  to  students,  teachers,  and  school  administrators.  Similar  to  the  development  of  the 
« cognitive  items,  the  development  of  the  policy  issues  and  questionnaire  items  was  a consensual  process 
that  involved  staff  work,  field  testing,  and  review  by  external  advisory  groups.  A Background 
Questionnaire  Panel  drafted  a set  of  policy  issues  and  made  recommendations  regarding  the  design  of  the 
items.  They  were  particularly  interested  in  capitalizing  on  the  unique  properties  of  NAEP  and  not 
duplicating  other  surveys  (e.g.,  the  National  Survey  of  Public  and  Private  School  Teachers  and 
Administrators,  the  School  and  Staffing  Survey,  and  the  National  Educational  Longitudinal  Study). 

The  Panel  recommended  a focused  study  that  addressed  the  relationship  between  student 
achievement  and  instructional  practices.  The  policy  issues,  items,  and  field  test  results  were  reviewed  by 
the  group  of  external  consultants  who  identified  specific  items  to  be  included  in  the  final  questionnaires. 
In  addition,  the  Mathematics  Instrument  Development  Panel  and  state  representatives  were  consulted  on 
the  appropriateness  of  issues  addressed  in  the  questionnaires  as  they  relate  to  mathematics  instruction 
and  achievement.  The  items  underwent  internal  ETS  review  procedures  to  ensure  fairness  and  quality  and 
were  then  assembled  into  questionnaires. 


=>  Student  Questionnaires 

In  addition  to  the  cognitive  items,  the  1996  assessment  included  three  student  questionnaires;  two 
sets  of  general  and  mathematics  background  items  designed  to  gather  contextual  information  about 
students,  their  instructional  experiences  in  mathematics,  and  their  attitudes  toward  mathematics,  and  one 
set  of  background  items,  given  to  students  at  the  end  of  each  booklet  to  determine  their  motivation  in 
completing  the  assessment  and  their  familiarity  with  assessment  tasks.  In  order  to  ensure  that  all  fourth- 
grade  students  understood  the  items  and  had  every  opportunity  to  respond  to  them,  the  three 
questionnaires  were  read  aloud  by  administrators  as  fourth-grade  students  read  along  and  responded  in 
their  booklets.  Background  questionnaires  were  not  read  aloud  to  eighth-  and  twelfth-grade  students. 

The  Student  Demographics  (common  background)  Questionnaire  included  items  about 
race/ethnicity,  language  spoken  in  the  home,  mother’s  and  father’s  level  of  education,  reading  materials 
in  the  home,  homework,  attendance,  which  parents  live  at  home,  and  which  parents  work.  This 
questionnaire  was  the  first  section  in  every  booklet.  In  many  cases  the  items  used  were  continued  from 
prior  assessments,  so  as  to  document  changes  in  contextual  factors  that  occur  over  time. 

The  second  section  of  background  items  was  the  Mathematics  Background  Questionnaire. 
Categories  of  information  represented  in  this  section  include: 

Time  Spent  Studying  Mathematics:  Students  were  asked  to  describe  both  the  amount  of 
instruction  they  received  in  mathematics  and  the  time  spent  on  mathematics  homework. 

Instructional  Practices:  Students  were  asked  to  report  their  instructional  experiences 
related  to  mathematics  in  the  classroom,  including  group  work,  special  projects,  and  writing 
in  response  to  mathematics.  In  addition,  they  were  asked  about  the  instructional  practices  of 
their  mathematics  teachers  and  the  extent  to  which  the  students  themselves  discussed  what 
they  did  in  class  and  demonstrated  use  of  skills  and  strategies. 
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Attitudes  Towards  Mathematics:  Students  were  asked  a series  of  questions  about  their 
attitudes  and  perceptions  about  mathematics. 

The  Student  Motivation  Questionnaire  asked  students  to  describe  how  hard  they  tried  on  the 
NAEP  mathematics  assessment,  how  difficult  they  found  the  assessment,  how  many  items  they  thought 
they  got  right,  how  important  it  was  for  them  to  do  well,  and  how  familiar  they  were  with  the  assessment 
format. 


=>  Teacher,  School,  and  SD/LEP  Student  Questionnaires 

To  supplement  the  information  on  instruction  reported  by  students,  the  mathematics  teachers  of 
the  students  participating  in  the  mathematics  assessment  were  asked  to  complete  a questionnaire  about 
their  instructional  practices,  teaching  backgrounds,  and  characteristics.  The  teacher  questionnaire 
contained  two  parts.  The  first  part  pertained  to  the  teachers’  background  and  general  training.  The  second 
part  pertained  to  specific  training  in  teaching  mathematics  and  the  procedures  the  teacher  uses  for  each 
class  containing  an  assessed  student,  as  well  as  collecting  information  on  teachers’  awareness  and 
knowledge  of  the  NCTM  Standards, 

The  Teacher  Questionnaire,  Part  I:  Background  and  General  Training  included  questions 
pertaining  to  gender,  race/ethnicity,  years  of  teaching  experience,  certification,  degrees,  major  and  minor 
fields  of  study,  course  work  in  education,  course  work  in  specific  subject  areas,  amount  of  in-service 
training,  extent  of  control  over  instructional  issues,  and  availability  of  resources  for  their  classroom. 

The  Teacher  Questionnaire,  Part  II:  Training  in  Mathematics  and  Classroom  Instructional 
Information  included  questions  on  the  teacher’s  exposure  to  various  issues  related  to  mathematics  and 
teaching  mathematics  through  pre-  and  in-service  training,  ability  level  of  students  in  the  class,  whether 
students  were  assigned  to  the  class  by  ability  level,  time  on  task,  homework  assignments,  frequency  of 
instructional  activities  used  in  class,  methods  of  assessing  student  progress  in  mathematics,  instructional 
emphasis  given  to  the  mathematics  abilities  covered  in  the  assessment,  and  use  of  particular  resources. 

A School  Characteristics  and  Policies  Questionnaire  was  given  to  the  principal  or  other 
administrator  of  each  school  that  participated  in  the  assessment.  This  information  provided  an  even 
broader  picture  of  the  instructional  context  for  students’  mathematics  achievement.  This  questionnaire 
included  questions  about  background  and  characteristics  of  school  principals,  length  of  school  day  and 
year,  school  enrollment,  absenteeism,  dropout  rates,  size  and  composition  of  teaching  staff,  policies 
about  grouping  students,  curriculum,  testing  practices  and  uses,  special  priorities  and  school-wide 
programs,  availability  of  resources,  special  services,  community  services,  policies  for  parental 
involvement,  and  school-wide  problems. 

The  SD/LEP  Student  Questionnaire  was  completed  by  the  teachers  of  students  who  were 
selected  to  participate  in  the  assessment  sample  who  were  also  identified  as  students  with  a disability 
(SD)  or  categorized  as  being  of  limited  English  proficiency  (LEP).  Some  of  these  students  were 
determined  by  the  school  to  be  ineligible  to  be  assessed.  In  order  to  be  excluded  from  the  assessment,  a 
student  must  have  been  identified  as  SD  and  must  not  have  been  mainstreamed  at  least  50  percent  of  the 
time,  or  was  categorized  as  LEP.  In  addition,  the  school  staff  would  have  needed  to  determine  that  it  was 
inappropriate  to  include  the  student  in  the  assessment.  This  questionnaire  asked  about  the  nature  of  the 
student’s  disability  or  about  the  student’s  native  language,  and  the  special  programs  in  which  the  student 
participated. 
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2.5 


DEVELOPING  THE  SCIENCE  ASSESSMENT 


2.5.1  Overview 

The  science  framework  for  the  1996  National  Assessment  of  Educational  Progress  (NAEP)  was 
produced  under  the  auspices  of  the  National  Assessment  Governing  Board  (NAGB).  The  consensus 
process  was  managed  by  the  Council  of  Chief  State  School  Officers  (CCSSO)  who  worked  with  the 
National  Center  for  Improving  Science  Education  and  the  American  Institutes  for  Research.  Items  were 
developed  that  were  aligned  with  the  specifications  described  in  the  framework  and  were  extensively 
reviewed  by  specialists  in  science,  measurement,  and  bias/sensitivity,  as  well  as  by  government  officials 
and  state  representatives. 

The  following  sections  include  a detailed  description  of  the  development  of  the  framework, 
objectives,  and  items  for  the  1996  NAEP  science  assessment.  Section  2.5.8  describes  the  student 
background  questionnaires  and  the  science  teacher  questionnaire.  Additional  information  on  the  structure 
and  content  of  assessment  booklets  can  be  found  in  Chapter  4.  Various  committees  worked  on  the 
development  of  the  framework,  objectives,  and  items  for  the  mathematics  assessment.  The  list  of 
committee  members  and  consultants  who  participated  in  the  1996  development  process  is  provided  in 
Appendix  A. 


2.5.2  Development  of  the  Assessment  Framework 

NAGB  is  responsible  for  setting  policy  for  NAEP;  this  policymaking  role  includes  the 
development  of  assessment  frameworks  and  test  specifications.  Appointed  by  the  Secretary  of  Education 
from  lists  of  nominees  proposed  by  the  Board  itself  in  various  statutory  categories,  the  26-member  board 
is  composed  of  state,  local,  and  federal  officials,  as  well  as  educators  and  members  of  the  public. 

The  science  framework  for  the  NAEP  1996  assessment  was  developed  over  a 10-month  period 
between  October  1990  and  August  1991.  The  following  sections  discuss  how  the  specifications  and  items 
for  science  assessment  were  developed.  The  assessment  instrument,  the  student  assessment  booklets,  and 
the  student,  teacher,  school,  and  SD/LEP  questionnaires  are  also  described. 

A consensus  process  run  by  CCSSO  was  used  to  produce  the  science  framework.  This  process 
involved  two’committees:  a Planning  Committee  that  conducted  much  of  the  actual  framework 
development  and  a steering  committee  that  provided  policy  and  general  guidance  for  the  project.  As 
general  guidelines  for  the  Planning  Committee,  the  Steering  Committee  that  recommended  that  the 
framework  and  ensuing  science  assessment  have  the  following  five  characteristics: 

• The  framework  should  reflect  the  best  thinking  about  the  knowledge,  skills,  and 
competencies  needed  for  a high  degree  of  scientific  understanding  among  all 
students  in  the  United  States.  Accordingly  it  should  encompass  knowledge  and  use 
of  organized  factual  information,  relationships  among  concepts,  major  ideas  unifying 
the  sciences,  and  thinking  and  laboratory  skills.  In  addition,  the  framework  should  be 
based  on  current  understandings  from  research  of  teaching,  learning,  and  students’ 
performance  in  science. 

• The  framework  and  the  assessment  should  address  the  nature  and  practices  of 
knowing  in  science,  as  different  from  other  ways  of  knowing;  reflect  the  quantitative 
aspects  of  science  as  well  as  the  concepts  of  life,  earth,  and  physical  sciences;  deal 
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with  issues  raised  by  the  role  of  science  and  technology  in  society;  include  practical 
problem  solving  in  science;  take  into  account  the  developmental  levels  of  students; 
and  ensure  that  students  with  diverse  backgrounds  are  assessed  in  ways  that  provide 
them  with  equal  and  fair  opportunities  to  reflect  their  knowledge  and  performance. 

• Assessment  formats  should  be  used  that  are  consistent  with  the  objectives  being 
assessed.  A variety  of  strategies  for  assessing  student  performance  are  advocated, 
including  performance  tasks  that  allow  students  to  manipulate  physical  objects  and 
draw  scientific  understandings  from  the  materials  before  them;  constructed-response 
items  that  provide  insights  into  students’  levels  of  understanding  and  ability  to 
communicate  in  the  sciences,  as  well  as  their  ability  to  generate,  rather  than  simply 
recognize  information  related  to  scientific  concepts  and  their  interconnections;  and 
multiple-choice  items  that  probe  students’  conceptual  understanding  and  ability  to 
connect  ideas  in  a scientifically  sound  way. 

• The  assessment  should  contain  a broad  enough  range  of  items  at  different  levels  of 
proficiency  for  identifying  three  achievement  levels  for  each  grade. 

• Information  on  students’  demographic  and  other  background  characteristics  should 
be  collected.  Additional  information  should  be  collected  from  students,  teachers  and 
administrators  about  instructional  programs  and  delivery  systems,  so  that  their 
relationships  with  student  achievement  can  be  ascertained  and  used  to  inform 
program  and  policy  decisions. 

A Planning  Committee  was  established  to  identify  goals  and  objectives  and  to  produce  the 
framework.  This  Planning  Committee  met  monthly  from  November  1990  through  April  1991  and  was 
joined  in  the  first  meeting  and  final  meeting  by  the  Steering  Committee,  which  reviewed  and  reacted  to 
all  framework  drafts.  During  this  development  process,  input  and  reactions  were  continually  sought  from 
a wide  range  of  committee  members  both  within  the  field  of  science  and  external  to  it.  A list  of 
committee  members  who  participated  in  the  developmental  process  is  provided  in  Appendix  A. 

2.5.3  Framework  for  the  1996  Assessment 

The  framework  for  the  1996  science  assessment  is  represented  as  a matrix  with  two  dimensions 
represented  by  three  fields  of  science  {earth  science^  physical  science,  and  life  science)  and  three 
elements  of  knowing  and  doing  science  ( conceptual  understanding,  scientific  investigation,  and 
practical  reasoning).  In  addition,  there  are  two  overarching  domains  that  describe  science  and  nature  of 
science  and  themes.  Figures  2-4  to  2-6,  respectively,  describe  the  three  fields  of  science,  the  elements  of 
knowing  and  doing  science,  and  the  overarching  domains. 
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Figure  2-4 

Descriptions  of  the  Three  Fields  of  Science 


Earth  Science 

The  earth  science  component  assessed  centers  on  objects  and  events  that  are  relatively  accessible 
or  visible.  The  concepts  and  topics  covered  are  solid  earth  (lithosphere),  water  (hydrosphere),  air 
(atmosphere),  and  the  earth  in  space.  The  solid  earth  consists  of  composition;  forces  that  alter  its  surface; 
the  formation,  characteristics,  and  uses  of  rocks;  the  changes  and  uses  of  soil;  natural  resources  used  by 
humankind;  and  natural  forces  within  the  earth.  Concepts  and  topics  related  to  water  consist  of  the  water 
cycle;  the  nature  of  oceans  and  their  effects  on  water  and  climate;  and  the  location  of  water,  its 
distribution,  characteristics,  and  effect  of  and  influence  on  human  activity.  The  air  is  broken  down  into 
composition  and  structure  of  the  atmosphere  (including  energy  transfer);  the  nature  of  weather;  common 
weather  hazards;  and  air  quality  and  climate.  The  earth  in  space  consists  of  setting  of  the  earth  in  the 
solar  system;  the  setting  and  evolution  of  the  solar  system  in  the  universe;  tools  and  technology  that  are 
used  to  gather  information  about  space;  apparent  daily  motions  of  the  sun,  the  moon,  the  planets  and  the 
stars;  rotation  of  the  earth  about  its  axis,  the  earth’s  revolution  around  the  sun;  and  tilt  of  the  earth’s  axis 
that  produces  seasonal  variations  in  the  climate. 

Physical  Science 

The  physical  science  component  relates  to  basic  knowledge  and  understanding  concerning  the 
structure  of  the  universe  as  well  as  the  physical  principles  that  operate  within  it.  The  major  sub-topics 
probed  are  matter  and  its  transformations,  energy  and  its  transformations,  and  the  motion  of  things. 
Matter  and  its  transformations  are  described  by  diversity  of  materials  (classification  and  types  and  the 
particulate  nature  of  matter);  temperature  and  states  of  matter;  properties  and  uses  of  material  (modifying 
properties,  synthesis  of  materials  with  new  properties);  and  resource  management.  Energy  and  its 
transformations  involve  different  forms  of  energy;  energy  transformations  in  living  systems,  natural 
physical  systems,  and  artificial  systems  constructed  by  humans;  and  energy  sources  and  use,  including 
distribution,  energy  conversion,  and  energy  costs  and  depletion.  Motion  is  broken  down  into  an 
understanding  of  frames  of  reference;  forces  and  changes  in  position  and  motion;  action  and  reaction; 
vibrations  and  waves  as  motion;  general  wave  behavior;  electromagnetic  radiation;  and  the  interactions 
of  electromagnetic  radiation  with  matter. 


Life  Science 

The  fundamental  goal  of  life  science  is  to  attempt  to  understand  and  explain  the  nature  and 
function  of  living  things.  The  major  concepts  assessed  in  life  science  are  change  and  evolution,  cells  and 
their  functions,  organisms,  and  ecology.  Change  and  evolution  includes  diversity  of  life  on  earth;  genetic 
variation  within  a species;  theories  of  adaptation  and  natural  selection;  and  changes  in  diversity  over 
time.  Cells  and  their  functions  consists  of  information  transfer;  energy  transfer  for  the  construction  of 
proteins;  and  communication  among  cells.  Organism  are  described  by  reproduction,  growth  and 
development;  life  cycles;  and  functions  and  interactions  of  systems  within  organisms.  The  topic  of 
ecology  centers  on  the  interdependence  of  life — populations,  communities,  and  ecosystems. 
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Figure  2-5 

Descriptions  of  Knowing  and  Doing  Science 

Conceptual  Understanding 

Conceptual  understanding  includes  the  body  of  scientific  knowledge  that  students  draw  upon 
when  conducting  a scientific  investigation  or  engaging  in  practical  reasoning.  Essential  scientific 
concepts  involve  a variety  of  information  including  facts  and  events  the  student  learns  from  science 
instruction  and  experiences  with  the  natural  environment  and  scientific  concepts,  principles,  laws,  and 
theories  that  scientists  use  to  explain  and  predict  observations  of  the  natural  world. 

Scientific  Investigation 

Scientific  investigation  probes  students’  abilities  to  use  the  tools  of  science,  including  both 
cognitive  and  laboratory  tools.  Students  should  be  able  to  acquire  new  information,  plan  appropriate 
investigations,  use  a variety  of  scientific  tools,  and  communicate  the  results  of  their  investigations. 

Practical  Reasoning 

Practical  reasoning  probes  students’  ability  to  use  and  apply  science  understanding  in  new,  real- 
word  applications. 


Figure  2-6 

Descriptions  of  Overarching  Domains 

The  Nature  of  Science 

The  nature  of  science  incorporates  the  historical  development  of  science  and  technology,  the 
habits  of  mind  that  characterize  these  fields,  and  methods  of  inquiry  and  problem-solving.  It  also 
encompasses  the  nature  of  technology  that  includes  issues  of  design,  application  of  science  to  real-world 
problems,  and  trade-offs  or  compromises  that  need  to  be  made. 

(continued) 
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Figure  2-6  (continued) 

Descriptions  of  Overarching  Domains 

Themes 

Themes  are  the  “big  ideas”  of  science  that  transcend  the  various  scientific  disciplines  and  enable 
students  to  consider  problems  with  global  implications.  The  NAEP  science  assessment  focuses  on  three 
themes:  systems,  models,  and  patterns  of  change. 

• Systems  are  complete,  predictable  cycles,  structures  or  processes  occurring  in  natural  phenomena. 
Students  should  understand  that  a system  is  an  artificial  construction  created  to  represent,  or  explain 
a natural  occurrence.  Students  should  be  able  to  identify  and  define  the  system  boundaries,  identify 
the  components  and  their  interrelationships  and  note  the  inputs  and  outputs  to  the  system. 

• Models  of  objects  and  events  in  nature  are  ways  to  understand  complex  or  abstract  phenomena.  As 
such  they  have  limits  and  involve  simplifying  assumptions  but  also  possess  generalizability  and  often 
predictive  power.  Students  need  to  be  able  to  distinguish  the  idealized  model  from  the  phenomenon 
itself  and  to  understand  the  limitations  and  simplified  assumptions  that  underlie  scientific  models. 

• Patterns  of  change  involve  students’  recognition  of  patterns  of  similarity  and  differences,  and 
recognition  of  how  these  patterns  change  over  time.  In  addition,  students  should  have  a store  of 
common  types  of  patterns  and  transfer  their  understanding  of  a familiar  pattern  of  change  to  a new 
and  unfamiliar  one. 


Table  2-6  summarizes  the  distribution  of  assessment  time  across  the  three  fields  of  science — 
earthy  physical^  and  life.  These  fields  provide  the  basis  for  the  content  area  scales. 


Table  2-6 

Percentage  Distribution  of  Items  by  Grade  and  Field  of  Science 
as  Specified  in  the  NAEP  Science  Framework 


Field  of  Science 

Grade  4 

Grade  8 

Grade  12 

Earth  Science 

33% 

30% 

33% 

Physical  Science 

33% 

30% 

33% 

Life  Science 

33% 

40% 

33% 
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Table  2-7  shows  the  distribution  of  assessment  time  by  knowing  and  doing  science. 


Table  2-7 

Percentage  Distribution  of  Items  by  Grade  and  Knowing  and  Doing  Science 
as  Specified  in  the  NAEP  Science  Framework 


Knowing  and  Doing  Science  Elements 

Grade  4 

Grade  8 

Grade  12 

Conceptual  Understanding 

45% 

45% 

45% 

Scientific  Investigation 

45% 

30% 

30%. 

Practical  Reasoning 

10% 

25% 

25% 

A number  of  items  that  assess  each  of  the  fields  of  science  and  each  of  the  ways  of  knowing  and 
doing  science  also  probe  nature  of  science  and  themes  {systems,  models,  and  patterns  of  change).  Table 
2-8  shows  the  recommended  and  actual  percentages  of  assessment  time  for  these  two  overarching 
domains. 


Table  2-8 

Percentage  Distribution  of  Items  Devoted  to  Nature  of  Science  and  Themes 
as  Specified  in  the  NAEP  Science  Framework 


Overarching  Domains 

Grade  4 

Grade  8 

Grade  12 

Nature  of  Science 

>15% 

>15% 

>15% 

Themes 

33% 

50% 

50% 

In  addition  to  calling  for  coverage  of  the  content  and  cognitive  domains  described  above,  the 
framework  instructed  that  all  students  participating  in  NAEP  take  part  in  a scientific  investigation  or 
hands-on  experiment.  In  addition,  it  indicated  that  at  least  30  percent  of  students  should  complete 
portions  of  the  assessment  involving  in-depth  examination  of  certain  themes  or  topics  in  science. 


2.5.4  Developing  the  Cognitive  Items 

The  1996  assessment  was  designed  to  meet  the  content  specifications  elaborated  in  the 
framework.  Because  of  the  broad  content  and  skills  definitions  included  in  the  framework,  and  the  need 
to  assess  hand-on  and  theme-based  science  skills,  the  exercise  development  effort  was  extensive.  At  each 
grade,  enough  blocks  were  field  tested  to  support  13  operational  blocks;  each  block  was  20  minutes  at 
grade  4 and  30  minutes  at  the  older  two  levels.®  In  addition,  other  portions  of  the  1993  field  test  were 
used  to  develop  exercises  for  a special  study  of  students  at  grade  twelve  who  had  received  advanced 
science  training.  Given  the  fact  that  the  science  assessment  was  delayed  from  1994  until  1996,  a 
supplemental  field  test  was  held  in  1995  (exercises  were  developed  for  this  field  test  in  1994).  This  round 
of  exercise  development  was  used  to  build  extra  exercises  for  the  general  science  and  advanced 
components  of  the  assessment. 

The  development  of  cognitive  items  involved  careful  field  testing,  both  locally  and  nationally,  of 
grade-appropriate  questions  and  tasks  for  the  assessment.  Items  were  selected  from  a pool  of  questions 


^ Most  of  the  development  and  field  testing  were  conducted  during  1992  and  1993;  supplemental  development  and  field  testing 
of  general  science  blocks  was  conducted  during  1994  and  1995. 


that  were  written  by  teachers  from  across  the  country  as  well  as  by  science  assessment  specialists  on  staff 
at  ETS.  The  framework  stated  that  the  assessment  should  include  some  performance-based  questions  and 
tasks  that  require  students  to  reason  and  solve  problems  representing  real-life  applications  of  science. 
Final  selections  of  items  used  in  the  1996  assessment  were  approved  by  the  Science  Instrument 
Development  Committee. 

The  assessment  included  constructed-response  (short  and  extended)  and  multiple-choice  items. 
The  decision  to  use  a specific  item  type  was  based  on  a consideration  of  the  most  appropriate  format  for 
assessing  the  particular  objective.  Both  types  of  constructed-response  items  were  designed  to  provide  an 
in-depth  view  of  students’  ability  to  communicate  their  understanding  of  important  concepts  in  science. 
Short  constructed-response  items  (scored  with  either  a 2-  or  3-level  scoring  rubric)  were  used  when 
students  needed  to  respond  briefly  in  order  to  demonstrate  full  comprehension.  Extended  constructed- 
response  items  (scored  with  a 4-  or  5-level  scoring  rubric)  were  used  when  the  task  required  more 
thoughtful  consideration  of  the  problem  and  engagement  in  more  complex  reasoning  processes.  Some 
items  also  required  diagrams,  graphs,  or  calculations.  It  was  expected  that  students  could  adequately 
answer  the  short  constructed-response  items  in  about  two  to  three  minutes  and  the  extended  constructed- 
response  items  in  about  five  minutes.  In  addition,  blocks  of  items  were  developed  that  required  the 
manipulation  of  equipment  (hands-on  tasks)  and  others  were  developed  that  assessed  each  of  the  three 
themes:  systems,  models,  and  patterns  of  change.  In  the  case  of  some  of  the  hands-on  blocks,  compound 
items  were  created  in  which  student  responses  to  a variety  of  items  were  scored  as  a single  item. 
Multiple-choice  items  were  used  when  a straightforward,  single  correct  answer  was  required. 

A carefully  developed  and  proven  series  of  steps  was  used  to  create  the  assessment  items.  These 
steps  are  described  earlier  in  the  chapter  under  Section  2.3. 


2.5.5  Development  of  the  Operational  Forms 

Most  of  the  items  for  the  1996  science  assessment  were  field  tested  in  February  and  March  1993; 
however,  since  the  assessment  was  delayed  from  1994  to  1996  an  opportunity  was  afforded  for  further 
items  to  be  field  tested  in  February  and  March  1995.  Each  of  these  field  tests  involved  students  in  many 
states  and  were  intended  to  try  out  the  cognitive  items  and  hands-on-tasks  and  to  give  jurisdictions  and 
contractors  practice  and  experience  with  the  proposed  materials  and  tasks.  Approximately  500  responses 
were  obtained  for  each  item  in  each  field  test. 

The  field  test  data  were  collected,  scored,  and  analyzed  in  preparation  for  meetings  with  the 
Science  Instrument  Development  Committee.  The  objectives  that  guided  the  review  of  these  items  were: 

• to  determine  which  items  were  most  suitable  for  assessing  understanding  in  science 
in  accordance  with  the  framework; 

• to  select  items  that  displayed  appropriate  statistical  attributes; 

• to  determine  the  need  for  revisions  of  items  that  lacked  clarity,  or  had  ineffective 
item  formats; 

• to  determine  appropriate  timing  for  assessment  items. 

Committee  members,  ETS  assessment  staff,  and  NAEP  staff  reviewed  the  materials.  Item 
analyses  (which  provided  the  percentage  of  correct  responses,  the  biserial  correlations  for  multiple- 
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choice  and  items  with  a two-level  scoring  guides,  and  percentages  of  responses  in  each  category  or  at 
each  level  of  the  scoring  guide  and  the  polyserial  correlations  for  other  constructed-response  items)  were 
used  as  a guide  in  identifying  and  flagging  for  further  review  those  test  items  that  were  not  measuring  the 
intended  objective  well. 

Once  the  committees  had  selected  the  items,  they  were  rechecked  for  content,  measurement,  and 
sensitivity  concerns.  In  addition,  a meeting  of  representatives  from  state  education  agencies  was 
convened  to  review  the  items  chosen  for  the  components  of  the  operational  assessment  that  were  to  be 
administered  at  the  state  level  (that  is,  the  grade  8 assessment).  The  federal  clearance  package  containing 
13  blocks  of  cognitive  items  was  submitted  to  NCES  in  August  1993.  A further  clearance  package 
containing  two  blocks  of  items  was  submitted  to  NCES  in  1995.  Throughout  the  clearance  process, 
revisions  were  made  in  accordance  with  changes  required  by  the  government.  Upon  approval,  the  15 
blocks  (assembled  into  booklets)  and  questionnaires  were  ready  for  printing  in  preparation  for  the 
assessment. 


2.5.6  Distribution  of  Assessment  Items 


The  science  assessment  developed  for  use  in  1996  was  organized  according  to  a series  of  blocks, 
each  containing  a set  of  items.  Some  of  the  blocks  were  unique  to  a particular  grade  level.  Other  blocks 
were  designed  to  be  given  to  students  in  two  grades  (either  4 and  8 or  8 and  12). 

At  each  grade,  the  main  component  of  the  assessment  used  for  the  creation  of  reporting  scales 
included  fifteen  different  blocks  of  multiple-choice  and  constructed-response  items  (some  with  both 
regular  and  extended  constructed-response  items).  These  blocks  each  were  20-minutes  long  at  grade  four 
and  thirty  minutes  at  the  other  levels.  Three  30-minute  special  study  blocks  targeted  students  in  advanced 
mathematics  courses.  Of  the  15  blocks  at  each  grade  that  were  used  for  the  main  assessment; 

• four  blocks  at  each  grade  level  required  students  to  complete  a hands-on  science  task 

• three  blocks  at  each  grade  required  students  to  work  in-depth  in  a single  thematic  or 
topical  area 

• eight  blocks  at  each  grade  covered  general  knowledge  and  concepts  in  the  fields  of 
science 

Figure  2-7  lists  the  total  number  of  items  at  each  grade  level  in  the  1996  assessment.  Of  the  total 
of  438  items,  there  were  167  unique  multiple-choice  items  and  271  unique  constructed-response  items 
that  made  up  the  1996  science  assessment.  Some  of  these  items  were  used  at  more  than  one  grade  level. 
As  a result,  the  sum  of  the  items  that  appear  at  each  grade  level  is  greater  than  the  total  number  of  unique 
items. 
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Figure  2-7 

Total  Number  of  Items  for  the  1996  Science  Main  Assessment 


10  Multiple-Choice  21  Multiple-Choice 

19  Short  Constructed- Response  24  Short  Construct^ -Response 

4 Extended  Constructed- Response  4 Extended  Constructed- Response 


* The  percentage  of  time  for  multiple-choice  items  is  low  because  the  hands-on  tasks  and  theme  blocks  contain  very  few 
multiple-choice  items  and  take  up  47%  of  the  assessment  time  at  each  grade  level. 


Table  2-9  summarizes  the  distribution  of  assessment  time  across  the  three  fields  of  science.  Since 
these  fields  make  up  the  core  of  the  Science  Framework,  care  has  been  taken  to  ensure  the  greatest  possible 
congruence  between  the  proportions  used  in  the  assessment  and  those  indicated  in  the  assessment 
specifications. 


Table  2-9 

Distribution  of  Assessment  Time  by  Field  of  Science 


Earth  Science 

Physical  Science 

Life  Science 

Grade  4 

33% 

34% 

33% 

Grade  8 

30% 

30% 

40% 

Grade  12 

33% 

■33% 

34% 
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Table  2-10  summarizes  the  assessment  in  terms  of  percentage  of  time  devoted  to  different  cognitive 
domains.  The  classification  of  items  into  these  domains  was  conducted  by  both  ETS  staff  and  members  of 
the  instrument  development  committee.  Every  effort  was  made  to  meet  the  specified  targets. 


Table  2-10 


Distribution  of  Assessment  Time  Across  Cognitive  Domains  (Knowing  and  Doing) 


Conceptual 

Understanding 

Scientific 

Investigation 

Practical 

Reasoning 

Grade  4 

45% 

38% 

17% 

Grade  8 

45% 

29% 

26% 

Grade  12 

44% 

28% 

28% 

Exercises  assessing  each  of  the  fields  of  science  and  each  of  the  cognitive  domains  also  probe 
three  themes,  (models,  systems,  and  patterns  of  change),  and  the  students’  knowledge  of  the  nature  of 
science.  Table  2-1 1 shows  the  distribution  of  assessment  time  devoted  to  themes,  and  Table  2-12 
shows  the  distribution  of  assessment  time  devoted  to  the  nature  of  science. 


Table  2-11 

Percentage  of  Assessment  Time  Devoted  to  Themes 


Actual 

Recommended 

Grade  4 

53%' 

30% 

Grade  8 

49% 

50% 

Grade  12 

55% 

50% 

'Several  of  the  hands-on  tasks  were  classified  as  themes. 


Table  2-12 

Percentage  of  Assessment  Time  Devoted  to  Nature  of  Science 


Actual 

Recommended 

Grade  4 

19% 

>15% 

Grade  8 

21% 

>15% 

Grade  12 

31% 

>15% 

In  addition  to  the  main  components  of  the  assessment,  an  advanced  study  was  conducted  at  grade 
12.  Students  who  were  in  their  fourth  year  of  high-school  science  were  sampled.  Each  participant  took  a 
linking  block  made  up  of  17  exercises  from  the  main  assessment,  and  three  special  advanced  blocks:  one 
in  biology,  one  in  chemistry,  and  one  in  physics.  Each  of  the  advanced  blocks  contained  seven  multiple- 
choice  and  nine  constructed-response  items. 
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2.5.7  Questionnaires 

As  part  of  the  national  assessment  (as  well  as  the  State  Assessment),  a series  of  questionnaires 
was  administered  to  students,  teachers,  and  school  administrators.  Similar  to  the  development  of  the 
cognitive  items,  the  development  of  the  policy  issues  and  questionnaire  items  was  a consensual  process 
that  involved  staff  work,  field  testing,  and  review  by  external  advisory  groups.  A Background 
Questionnaire  Committee  drafted  a set  of  policy  issues  and  made  recommendations  regarding  the  design 
of  the  items.  They  were  particularly  interested  in  capitalizing  on  the  unique  properties  of  NAEP  and  not 
duplicating  other  surveys  (e.g.,  the  National  Survey  of  Public  and  Private  School  Teachers  and 
Administrators,  the  School  and  Staffing  Study,  and  the  National  Educational  Longitudinal  Study).  The 
policy  issues,  items,  and  field  test  results  were  reviewed  by  the  group  of  external  consultants  who 
identified  specific  items  to  be  included  in  the  final  questionnaires.  In  addition,  the  Science  Instrument 
Development  Committee  and  state  representatives  were  consulted  on  the  appropriateness  of  issues 
addressed  in  the  questionnaires  as  they  relate  to  science  instruction  and  performance.  The  items 
underwent  internal  ETS  review  procedures  to  ensure  fairness  and  quality  and  were  then  assembled  into 
questionnaires.  The  questionnaires  were  then  submitted  to  the  Office  of  Management  and  Budget  (OMB) 
for  approval. 


=>  Student  Questionnaires 

In  addition  to  three  blocks  of  cognitive  items,  each  booklet  in  the  assessment  included  three 
student  questionnaires.  Two  of  these  were  sets  of  general  and  science  background  questionnaires 
designed  to  gather  contextual  information  about  students,  their  instructional  experiences  in  science,  and 
their  attitudes  toward  science.  The  third  questionnaire  was  given  to  students  at  the  end  of  each  booklet  to 
determine  students’  motivation  in  completing  the  assessment  and  their  familiarity  with  assessment  tasks. 

The  Student  Demographics  (common  background)  Questionnaire  included  questions  about 
race/ethnicity,  mother’s  and  father’s  level  of  education,  types  of  reading  materials  in  the  home,  and 
school  attendance. 

The  Science  Background  Questionnaire  included  questions  that  addressed  the  following. 

Attitudes  Towards  Sciences:  Students  were  asked  a series  of  questions  about  their  attitudes 
and  perceptions  about  science. 

Time  Spent  Studying  Science:  Students  were  asked  to  describe  both  the  amount  of 
instruction  they  received  in  science  and  the  time  spent  on  science  homework. 

Instructional  Practices:  Students  were  asked  to  report  their  instructional  experiences 
related  to  science  in  the  classroom,  including  group  work,  special  projects,  and  writing  in 
response  to  science.  In  addition,  they  were  asked  about  the  instructional  practices  of  their 
science  teachers. 


The  Student  Motivation  Questionnaire  asked  students  how  many  questions  they  thought  they 
got  right  on  the  NAEP  science  assessment,  how  difficult  they  found  it,  how  hard  they  tried,  how 
important  it  was  for  them  to  do  well,  and  how  often  they  wrote  long  answers  on  tests  or  assignments  for 
science. 
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Teacher,  School,  and  SD/LEP  Student  Questionnaires 

To  supplement  the  information  on  instruction  reported  by  students,  the  science  teachers  of  the 
students  participating  in  the  assessment  were  asked  to  complete  a questionnaire  that  addressed  teachers’ 
background  and  general  training  as  well  as  their  science  preparation  and  information  concerning  science 
instruction. 

The  Teacher  Questionnaire,  Part  I:  Background  and  General  Training  included  questions 
about  gender,  race/ethnicity,  years  of  teaching  experience,  certification,  degrees,  major  and  minor  fields 
of  study,  course  work  in  education,  course  work  in  specific  subject  areas,  amount  of  in-service  training, 
professional  development  activities,  and  availability  of  resources  for  their  classroom. 

The  Teacher  Questionnaire,  Part  II:  Science  Preparation  and  Science  Instructional 
Information  included  questions  on  the  number  and  types  of  science  courses  taken  over  the  past  two 
years,  membership  in  science  organizations,  frequency  of  instructional  activities  such  as  asking  students 
to  prepare  a written  science  report  or  an  oral  science  report,  emphasis  on  objectives  such  as  developing 
science  problem-solving  skills,  methods  used  to  assess  student  progress  in  science,  and  ability  level  of 
students  in  class. 

A School  Characteristics  and  Policies  Questionnaire  was  given  to  the  principal  of  each  school 
that  participated  in  the  assessment  program.  This  questionnaire  asked  about  background  and 
characteristics  of  school  principals,  length  of  school  day  and  year,  school  enrollment,  absenteeism,  drop- 
out rates,  size  and  composition  of  teaching  staff,  policies  about  grouping  students,  curriculum,  testing 
practices  and  uses,  special  priorities  and  school-wide  programs,  availability  of  resources,  special 
services,  community  services,  policies  for  parental  involvement,  and  school-wide  problems. 

The  SD/LEP  Student  Questionnaire  was  completed  by  the  teachers  of  those  students  who  were 
selected  to  participate  in  the  assessment  sample  and  were  identified  as  students  with  a disability  (SD)  or 
were  categorized  as  being  of  limited  English  proficiency  (LEP).  Some  of  these  students  were  determined 
by  the  school  to  be  ineligible  to  be  assessed.  In  order  to  be  excluded  from  the  assessment,  a student  must 
have  been  identified  as  SD  and  must  not  have  been  mainstreamed  at  least  50  percent  of  the  time,  or  was 
categorized  as  LEP.  In  addition,  the  school  staff  would  have  needed  to  determine  that  it  was 
inappropriate  to  include  these  students  in  the  assessment.  This  questionnaire  asked  about  the  nature  of 
the  student’s  disability  or  about  the  students’  native  language,  and  the  special  programs  in  which  the 
student  participated. 


Chapter  3 


SAMPLE  DESIGN^ 

Leslie  Wallace  and  Keith  F.  Rust 
Westat,  Inc. 


3.1  INTRODUCTION 

The  samples  for  the  1996  NAF.P  assessment  were  selected  using  a complex  multistage  sample 
design  involving  the  sampling  of  students  from  selected  schools  within  94  selected  geographic  areas, 
called  primary  sampling  units  (PSUs),  across  the  United  States. 

The  long-term  trend  sample  design  had  four  steps  in  the  selection  process  and  the  main  sample 
design  had  five  steps  in  the  selection  process: 

1 . selection  of  geographic  PSUs  (counties  or  groups  of  counties), 

2.  selection  of  schools  within  PSUs, 

3.  assignment  of  sample  type  to  schools  (main  samples  only), 

4.  assignment  of  session  types  to  schools,  and 

5.  selection  of  students  for  session  types  within  schools. 

The  samples  were  drawn  for  the  three  different  age  classes,^  and  for  each  age  class  the  samples 
were  of  two  distinct  types.  The  first  type  consisted  of  the  cross-sectional  or  “main”  samples,  while  the 
second  type  consisted  of  the  long-term  trend  samples.  The  populations  surveyed  with  each  of  these 
sample  types  are  defined  in  Table  1-1  in  Chapter  1.  Separate  samples  of  schools  were  required  for  the 
long-term  trend  samples  and  main  samples,  because  of  various  differences  in  the  calendar  period  for  test 
administration,  the  format  of  the  administration,  the  fact  that  the  trend  samples  include  age-based 
samples,  whereas  main  samples  do  not  and,  in  the  case  of  age  class  17,  the  grade  definition  of  the 
population  of  interest.  (See  the  description  of  Table  1-1  in  Chapter  1.) 

In  addition  to  representing  the  respective  populations  as  a whole,  for  the  main  samples  there  was 
oversampling  of  nonpublic  schools,  and  of  public  schools  with  moderate  or  high  enrollment  of  Black  or 
Hispanic  students  (see  Section  3.3).  This  oversampling  was  undertaken  to  increase  the  sample  sizes  of 
nonpublic-school  students  and  minority  students,  so  as  to  increase  the  reliability  of  estimates  for  these 
groups  of  students.  These  oversampling  rates  have  been  used  in  the  past  several  rounds  of  NAEP.  The 
oversampling  rates  were  based  on  experience,  after  attempting  to  report  results  for  these  groups  in 
assessments  where  no  oversampling  was  used. 


‘ Ralph  DiGaetano,  Keith  F.  Rust,  and  Leslie  Wallace  were  responsible  for  the  design  and  implementation  of  the  sampling 
process  for  the  1 996  NAEP  assessments, 

^ The  term  “age  class”  is  used  in  this  report  when  it  is  appropriate  to  discuss  one  of  the  three  student  cohorts  in  a general  way 
(not  necessarily  in  reference  to  a specific  sample).  For  the  1996  assessment,  age  class  9 refers  to  age  9 or  grade  4 long-term  trend, 
or  grade  4 main  sample  students;  age  class  13  refers  to  age  13  or  grade  8 long-term  trend,  or  grade  8 main  sample  students;  and 
age  class  17  refers  to  age  17  or  grade  1 1 long-term  trend,  or  grade  12  main  sample  students. 
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The  overall  assessment  period  fell  into  three  time  periods— fall,  winter,  and  spring.  Not  all 
assessment  components  were  conducted  in  each  time  period.  Table  3-1  shows  the  relationship  between 
the  various  sample  components  and  the  assessment  periods.  The  sizes  of  the  PSU  and  school  samples  and 
the  procedures  for  their  selection  were  determined  by  the  assessment  period,  as  well  as  by  the  population 
to  be  surveyed  and  the  method  of  administration  in  each  case. 

Table  3-1 


Assessment  Type  by  Age  Class  and  Assessment  Period 


Age  Class/Assessment 

Fall 

Winter 

Spring 

9 

Main 

— 

1/3/96  - 3/29/96 

_ 

Long-Term  Trend 

— 

1/3/96  - 3/8/96 



13 

Main 

— 

1/3/96  - 3/29/96 

_ 

Long-Term  Trend 

10/9/95  - 12/22/95 

— 



17 

Main 

— 

1/3/96  - 3/29/96 

_ 

Long-Term  Trend 

— 

— 

3/11/96-5/10/96 

Special  trend  samples  were  required  because: 

• The  long-term  trend  samples  had  different  school  and  student  eligibility 
requirements  than  the  main  samples.  Both  grade-  and  age-eligible  students  were 
targeted  in  long-term  trend,  and  only  grade-eligible  students  were  targeted  in  the 
main  samples.  This  meant  that  schools  with  any  of  several  grades  were  eligible  for 
long-term  trend  (grades  2 to  6 for  age  class  9,  grades  6 to  9 for  age  class  13,  and 
grades  9 to  12  for  age  class  17),  while  only  schools  with  grades  4,  8,  or  12  were 
eligible  for  the  main  samples. 

• The  conditions  for  administration  of  the  assessment  varied  considerably  between  the 
main  sample  and  long-term  trend  sample  sessions. 

• The  need  in  the  long-term  trend  samples  for  four  distinct  session  types  for  age  class 
9 and  13  and  three  for  age  class  17,  together  with  the  need  for  up  to  six  distinct 
session  types  for  the  main  samples,  made  it  not  feasible  to  conduct  both  main  sample 
sessions  and  long-term  trend  sessions  in  a given  school.  For  long-term  trend,  the 
session  types  were  spiral  booklets  51-56  and  tape  booklets  91-93  for  ages  9 and  13, 
and  spiral  booklets  51-56  and  tape  booklets  84-85  for  age  17.  For  the  main  samples, 
the  session  types  were  mathematics,  science,  mathematics  estimation,  and 
mathematics  theme  at  all  grades  (4,  8,  and  12),  advanced  mathematics  at  grades  8 
and  12,  and  advanced  science  at  grade  12. 

• For  age  classes  13  and  17,  the  main  sample  administrations  were  conducted  in  the 
winter;  while  the  long-term  trend  sample  administrations  were  conducted  in  the  fall 
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and  spring  respectively.  The  fall  and  spring  administration  periods  match 
administration  periods  used  in  NAEP  as  far  back  as  1969-71. 

This  chapter  gives  details  of  the  sample  selection  procedure,  and  information  on  the  results  of  the 
sampling  process.  Further  details  are  given  in  the  report  The  1996  NAEP  Sampling  and  Weighting  Report 
(Wallace  & Rust,  1999). 


3.2  PRIMARY  SAMPLING  UNITS 

In  the  first  stage  of  sampling,  the  United  States  (the  50  states  and  the  District  of  Columbia)  was 
divided  into  geographic  primary  sampling  units  (PSUs).  The  PSUs  are  those  that  were  used  beginning  in 
1994  and  incorporate  1990  U.S.  Census  information.  With  a few  exceptions,  each  PSU  met  a minimum 
size  requirement  (a  1990  U.S.  Census  population  of  at  least  60,000  in  the  Northeast  and  Southeast  and 
45,000  in  the  Central  and  West  regions)  and  comprised  either  a consolidated  metropolitan  statistical  area 
(CMSA),  a metropolitan  statistical  area  (MSA),  a single  county,  or  (more  likely  in  the  case  of  nonMSA 
PSUs)  a group  of  contiguous  counties.  In  the  case  of  New  England  MSAs,  which  are  not  formed  from 
whole  counties,  the  corresponding  New  England  County  Metropolitan  Areas  (NECMAs),  which  are 
defined  in  terms  of  whole  counties,  were  designated  as  PSUs.  The  PSUs  were  designed  to  serve  as  the 
PSUs  for  NAEP  samples  from  1994  until  2002.  Thus  1990  total  population  was  used  as  a size  measure, 
rather  than  1990  school  age  population,  as  this  was  considered  likely  to  correlate  more  highly  with  school 
age  population  over  this  period.  Each  PSU  was  contained  entirely  within  one  of  the  four  NAEP  regions 
defined  in  Table  3-2.  These  NAEP  regions  were  used  to  stratify  the  PSUs,  ensuring  that  each  region  was 
adequately  represented  in  the  various  assessment  samples. 


Table  3-2 

Geographic  Regions  Used  for  Stratification 


Northeast 

Southeast 

Central 

West 

Connecticut 

Alabama 

Illinois 

Alaska 

Delaware 

Arkansas 

Indiana 

Arizona 

District  of  Columbia 

Florida 

Iowa 

California 

Maine 

Georgia 

Kansas 

Colorado 

Maryland 

Kentucky 

Michigan 

Hawaii 

Massachusetts 

Louisiana 

Minnesota 

Idaho 

New  Hampshire 

Mississippi 

Missouri 

Montana 

New  Jersey 

North  Carolina 

Nebraska 

Nevada 

New  York 

South  Carolina 

North  Dakota 

New  Mexico 

Pennsylvania 

Tennessee 

Ohio 

Oklahoma 

Rhode  Island 

Virginia* 

South  Dakota 

Oregon 

Vermont 

Virginia^ 

West  Virginia 

Wisconsin 

Texas 

Utah 

Washington 

Wyoming 

' That  part  of  Virginia  that  is  part  of  the  Washington,  DC-MD-VA  metropolitan  statistical  area  at 
the  time  of  the  1990  Census,  is  included  in  the  Northeast  region;  the  remainder  of  the  state  is 
included  in  the  Southeast  region. 
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In  a few  cases  an  MSA  crossed  region  boundaries.  Such  MSAs  were  split  into  two  or  more  PSUs 
as  necessary  (e.g.,  the  Cincinnati  OH-KY-IN  MSA  was  split  into  the  Cincinnati  OH-IN  PSU  in  the 
Central  region  and  the  Cincinnati  KY  PSU  in  the  Southeast).  Ninety-four  PSUs  were  selected  for  the 
main  samples  and  52  PSUs  were  selected  for  the  long-term  trend  samples,  as  described  below. 

For  the  main  samples,  the  22  largest  PSUs  were  included  with  certainty.  The  inclusion  of  these 
PSUs  in  the  sample  with  certainty  provided  an  approximately  optimum,  cost-efficient  sample  of  schools 
and  students  when  samples  were  drawn  within  them  at  the  required  national  sampling  rate.  The  22  largest 
PSUs  by  region  were: 


22  Largest  PSUs  by  Region 


Northeast: 

> Baltimore,  MD  MSA 

> Boston-Lawrence-Salem-Lowell- 
Brockton,  MA  NECMA 

> New  York-Northern  New  Jersey-Long 
Island,  NY-NJ  CMS  A (excluding  that 
part  in  CT) 

> Philadelphia-Wilmington-Trenton, 
PA-NJ-DE-MD  CMSA 

> Pittsburgh-Beaver  Valley,  PA  CMSA 

> Washington,  DC-MD-VA  MSA 

Southeast: 

> Atlanta,  GA  MSA 

> Miami-Fort  Lauderdale,  FL  CMSA 

> Tampa-St.  Petersburg-Clearwater,  FL 
MSA 


Central: 

> Chicago-Gary-Lake  County,  IL-IN-WI  CMSA 

> Cleveland-Akron,  OH  CMSA 

> Detroit-Ann  Arbor,  MI  CMSA 

> Minneapolis-St.  Paul,  MN-WI  MSA 

> St.  Louis,  MO-IL  MSA 

West: 

> Dallas-Fort  Worth,  TX  CMSA 

> Denver-Boulder,  CO  CMSA 

> Houston-Galveston-Brazoria,  TX  CMSA 

> Los  Angeles-Anaheim-Riverside,  CA  CMSA 

> Phoenix,  AZ  MSA 

> San  Diego,  CA  MSA 

> San  Francisco-Oakland-San  Jose,  CA  CMSA 

> Seattle-Tacoma,  WA  CMSA 


The  remaining  smaller  PSUs  were  not  guaranteed  to  be  selected  for  the  sample.  These  were 
grouped  into  a number  of  noncertainty  strata  (so  called  because  the  PSUs  in  these  strata  were  not 
included  in  the  sample  with  certainty),  and  one  PSU  was  selected  from  each  stratum.  The  PSUs  were 
classified  into  four  regions,  each  containing  about  one-fourth  of  the  U.S.  population.  These  regions  were 
defined  primarily  by  state  (Table  3-2).  In  each  region,  noncertainty  PSUs  were  classified  as  MSA 
(metropolitan)  or  nonMSA  (nonmetropolitan)  according  to  1990  definitions.  The  resulting  major  strata 
are  shown  in  Table  3-3. 


Table  3-3 

The  Sampling  Major  Strata 
and  the  Number  of  Noncertainty  Strata  in  Each 


Region 

Number  of  Strata 
for  MSA  PSUs 

Number  of  Strata 
for  NonMSA  PSUs 

Total 

Strata 

Northeast 

6 

4 

10 

Southeast 

12 

12 

24 

Central 

8 

12 

20 

West 

10 

8 

18 

Total 

36 

36 

72 

58 

O 

ERIC 
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Within  each  major  stratum,  further  stratification  was  achieved  by  ordering  the  noncertainty 
PSUs  according  to  several  additional  socioeconomic  characteristics,  yielding  72  strata.  The  number  of 
such  strata  formed  within  each  major  stratum  is  shown  in  Table  3-3.  The  strata  were  defined  so  that  the 
aggregate  of  the  measures  of  size  of  the  PSUs  in  a stratum  was  approximately  equal  for  each  stratum.  The 
size  measure  used  was  the  population  from  the  1990  Census.  The  characteristics  used  to  define  strata 
were  the  percent  minority  population,  the  percentage  change  in  total  population  since  1980,  the  per  capita 
income,  the  percent  of  persons  age  25  or  over  with  college  degrees,  the  percent  of  persons  age  25  or  over 
who  completed  high  school,  and  the  civilian  unemployment  rate.  Up  to  four  of  these  characteristics  were 
used  in  any  one  major  stratum.  For  each  major  stratum,  the  characteristics  used  were  chosen  by  modeling 
PSU-level  mean  reading  proficiency  scores  for  1988,  1990,  and  1992.  The  characteristics  chosen  were 
the  best  predictors  of  PSU-level  mean  reading  proficiency  scores  in  these  models.  One  PSU  was  selected 
with  probability  proportional  to  size  from  each  of  the  72  noncertainty  strata.  That  is,  within  each  stratum, 
a PSU’s  probability  of  being  selected  was  proportional  to  its  population.  The  PSUs  were  selected  with 
probability  proportional  to  size  (PPS)  with  the  twin  aims  of  obtaining  approximately  self-weighting 
samples  of  students,  and  having  approximately  equal  workloads  in  each  PSU. 

Samples  of  94  PSUs  each  were  drawn  for  the  1994,  1996,  1998,  2000,  and  2002  main  samples 
simultaneously.  They  were  drawn  to  minimize  overlap  of  the  PSUs  from  one  assessment  to  the  next, 
except  that  certainty  PSUs  were  retained  in  each  assessment  year,  and  some  of  the  larger  noncertainty 
PSUs  are  in  the  sample  for  more  than  one  of  these  assessment  years.  Each  main  sample  of  94  PSUs  was 
drawn  from  a population  of  about  1,000  PSUs.  Primarily  because  of  the  use  of  MSAs  as  PSUs,  PSUs 
varied  considerably  as  to  their  probability  of  selection,  since  they  varied  greatly  in  size.  In  1996,  the  36 
selected  noncertainty  MSA  PSUs  had  probabilities  of  selection  ranging  from  0.04  to  0.58,  while  the  36 
selected  nonMSA  PSUs  had  probabilities  ranging  from  0.03  to  0.08.  Parts  of  43  states  were  included  in 
the  main  sample  PSUs. 

For  the  long-term  trend  samples,  52  PSUs  were  selected.  The  long-term  trend  samples  were  much 
smaller  than  the  main  samples  and  used  separate  field  staff.  Fewer  PSUs  were  used  for  the  long-term 
trend  samples  to  avoid  having  the  sample  spread  too  thinly  across  PSUs.  The  long-term  trend  PSUs  were 
drawn  for  1994,  1996,  1998,  2000,  and  2002  to  minimize  overlap  of  the  trend  PSUs  from  one  assessment 
to  the  next,  and  to  minimize  overlap  between  trend  and  main  samples  within  the  same  assessment. 

The  10  largest  main  sample  certainty  PSUs  were  also  included  with  certainty  in  the  long-term 
trend  samples.  Six  additional  PSUs  were  selected  systematically  and  with  probability  proportional  to  the 
1990  population  from  the  12  remaining  main  sample  certainties.  Finally,  36  PSUs  were  selected  from  the 
72  noncertainty  strata  so  that  the  overall  procedure  was  equivalent  to  systematic  sampling  with 
probabilities  proportional  to  the  1990  population.  The  72  noncertainty  strata  from  the  main  sample 
design  were  paired,  and  one  PSU  per  pair  was  selected  for  the  trend  samples.  Note  that  the  noncertainty 
long-term  trend  PSUs  are  not  a subsample  of  the  noncertainty  main  sample  PSUs,  in  order  to  minimize 
the  burden  on  a given  school  district  in  any  one  year. 


3.3  SELECTION  OF  SCHOOLS 

In  the  second  stage  of  sampling,  the  public  schools  (including  Bureau  of  Indian  Affairs  (BIA) 
schools  and  Department  of  Defense  Education  Activity  (DoDEA)  schools)  and  nonpublic  schools 
(including  Catholic  schools)  were  listed  according  to  the  grade  ranges  associated  with  the  three  age 
classes.  The  lists  of  schools  were  obtained  from  two  sources.  Regular  public,  BIA,  and  DoDEA  schools 
were  obtained  from  the  1994  list  of  schools  maintained  by  Quality  Education  Data,  Inc.  (QED).  Regular 
public  schools  are  schools  with  students  who  are  classified  as  being  in  a specific  grade  (as  opposed  to 


schools  having  only  “ungraded”  classrooms).  This  includes  statewide  magnet  schools  and  charter 
schools.  Catholic  and  other  nonpublic  schools  were  obtained  from  both  QED  and  the  Private  School 
Survey  (PSS)  developed  for  the  National  Center  for  Education  Statistics’  1993-1994  School  and  Staffing 
Survey.  The  majority  of  the  PSS  list  comes  from  complete  enumeration  of  schools,  but  a small  portion  of 
the  PSS  list  was  restricted  to  a sample  of  counties  selected  for  the  survey.  Certain  PSS  counties, 
generally  large  in  population,  were  also  included,  independently  by  chance,  in  the  NAEP  sample  PSUs. 
The  schools  from  such  counties  were  added  to  the  NAEP  sampling  frame  after  steps  were  taken  to 
eliminate  duplicates  with  the  QED  list  of  nonpublic  schools.  In  previous  years,  nonpublic  schools  were 
also  obtained  from  telephone  directories.  This  process  was  not  repeated  in  1996  because  the  use  of  the 
PSS  files  supplanted  the  need  for  this  supplement. 

Table  3-4  shows  the  numbers  of  schools  included  in  the  various  sampling  frame  components. 

The  population  of  eligible  schools  for  each  age  class  was  restricted  to  the  selected  PSUs.  Main  sample 
schools  were  selected  from  the  94  main  sample  PSUs  and  long-term  trend  schools  were  selected  from  the 
52  long-term  trend  PSUs.  Note  that  there  are  relatively  large  numbers  of  nonpublic  schools  that  are  listed 
in  the  QED  or  PSS  only.  The  discrepancy  between  the  schools  contained  in  the  PSS  dataset  versus  those 
in  the  QED  dataset  is  primarily  due  to  two  factors:  (1)  the  relative  outdatedness  of  the  two  school  lists, 
and  (2)  PSS’s  inclusion  of  a special  area  supplement  designed  to  find  schools  not  normally  available  on 
lists. 


Table  3-4 

Grade  Definition  of  School  Eligibility  for  Sampling  Frame  Inclusion 
and  Frame  Sizes,  Main  and  Long-Term  Trend  Samples^ 


Sample^ 

Sampling  Frame 
Included  Schools 
With  Any 
of  Grades 

Public^ 

Nonpublic 
from  QED 
Only“ 

Nonpublic 
from  PSS 
Only" 

Nonpublic 
from  QED 
and  PSS" 

Main  Samples 

Grade  4 

4 

18,046 

1,308 

1,716 

7,300 

Grade  8 

8 

6,093 

1,029 

1,322 

6,409 

Grade  12 

12 

4,357 

578 

869 

2,474 

Long-Term  Trend  Samples 

Age  Class  9 

2-5 

15,873 

1,147 

1,600 

6,019 

Age  Class  13 

6-9 

13,667 

1,089 

1,474 

6,592 

Age  Class  1 7 

9-12 

3,495 

' 491 

829 

2,106 

' The  numbers  in  this  table  reflect  the  full  samples,  including  all  sample  types  (see  Section  3.4). 

Age  Class”  is  a term  that  refers  to  either  an  age  or  a grade  definition  of  the  samples.  For  the  1996  main  assessments,  unlike 
for  previous  main  assessments,  only  grade  samples  were  drawn.  Lxrng-term  trend  age  class  definitions  vary  by  subject  area. 
They  are  explained  more  fully  in  Chapters  14  through  17. 

^Public,  BIA,  and  DoDEA  schools 
* Catholic  and  other  nonpublic  schools 


Any  school  having  one  or  more  of  the  eligible  grades,  and  located  within  an  appropriate  PSU, 
was  included  in  the  sampling  frame  of  schools  (the  list  of  schools  from  which  the  samples  of  schools 
were  drawn)  for  a given  sample.  For  each  age  class  in  the  long-term  trend  samples,  only  a fraction  of  one 
percent  of  age-eligible  students  was  enrolled  in  ineligible  schools.  An  independent  sample  of  schools  was 
selected  for  each  of  the  age  classes,  separately  for  main  and  long-term  trend.  Thus,  some  schools  were 
selected  for  assessment  of  two  age  classes,  and  a few  were  selected  for  all  three.  For  all  three  age  classes, 
a sample  of  schools  was  first  drawn  for  the  long-term  trend  assessments.  The  schools  selected  for 
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long-term  trend  at  a particular  age  class  were  excluded  from  the  sampling  frame  when  the  samples  of 
schools  for  the  corresponding  grade  were  drawn  for  the  main  assessments.  In  addition,  the  schools 
selected  for  the  1996  NAEP  State  Assessment  program  at  grade  8 were  excluded  from  the  sampling 
frame  for  the  main  samples  at  that  grade.  In  regard  to  both  of  these  situations,  adjustments  were  made  to 
the  sampling  weights  to  reflect  the  appropriate  probabilities  of  selection  to  yield  unbiased  estimates  for 
both  long-term  trend  and  main  samples. 

For  each  NAEP  sample,  schools  were  selected  (without  replacement)  across  all  PSUs  with 
probabilities  proportional  to  assigned  measures  of  size.  In  those  certainty  PSUs  included  in  both  main 
and  trend  samples,  the  probability  of  selection  for  long-term  trend  for  any  school  in  a given  age  class  was 
capped  at  0.5,  to  ensure  that  adequate  schools  remained  to  be  selected  for  the  main  sample.  For  long-term 
trend  samples,  the  measure  of  size  used  for  each  school  was  the  estimated  number  of  age  eligible  students 
in  the  school,  since  for  each  age  class  the  large  majority  of  students  selected  were  assigned  to  sessions 
for  which  only  students  of  the  appropriate  age  were  eligible.  In  most  schools  having  the  modal  grade, 
some  additional  students  were  selected  who  were  in  the  modal  grade  but  not  age-eligible,  so  that  the 
maximum  sample  size  of  students  within  a school  was  about  80  grade-  and  age-eligible  students.  Equal 
measures  of  size  were  assigned  to  schools  containing  estimates  of  age-eligible  students  ranging  from  20 
to  60  for  each  age  class.  Schools  with  more  than  60  age-eligible  students  were  selected  with  probabilities 
proportional  to  the  measure  of  size.  Schools  with  fewer  than  20  estimated  age-eligible  students  were 
assigned  somewhat  lower  measures  of  size,  and  thus  lower  probabilities  of  selection,  since  assessment  in 
these  schools  involved  substantially  higher  per-student  administrative  costs. 

For  the  main  samples,  equal  measures  of  size  were  assigned  to  schools  containing  estimates  of 
grade-eligible  students  ranging  from  20  to  120  (for  grade  4),  20  to  150  (for  grade  8),  or  20  to  180 
(for  grade  12).  Schools  larger  than  the  indicated  maximum  size  were  selected  with  probabilities 
proportional  to  the  measure  of  size.  This  procedure  was  used  so  as  to  obtain  approximately  self- 
weighting samples  of  students  (i.e.,  students  selected  with  approximately  equal  overall  probabilities)  at 
each  grade.  Three  variations  to  the  overall  goal  of  self-weighting  samples  were  implemented.  Schools 
with  fewer  than  20  estimated  grade-eligible  students  were  assigned  somewhat  lower  measures  of  size, 
and  thus  lower  probabilities  of  selection.  This  was  designed  to  increase  cost  efficiency.  Each  public 
school  designated  as  high-minority  (with  over  10  percent  Black  and/or  Hispanic  enrollment  for  grades  4 
and  8,  or  over  15  percent  Black  and/or  Hispanic  enrollment  for  grade  12)  was  given  double  the 
probability  of  selection  of  a public  school,  not  designated  high-minority,  of  similar  size  in  the  same  PSU. 
Such  high-minority  schools  were  oversampled  in  order  to  enlarge  the  sample  of  Black  and  Hispanic 
students,  thereby  enhancing  the  reliability  of  estimates  for  these  groups.  For  a given  overall  size  of 
sample,  this  procedure  reduces  somewhat  the  reliability  of  estimates  for  all  students  as  a whole  and  for 
those  not  Black  or  Hispanic.  Each  nonpublic  school  was  given  triple  the  probability  of  selection  of  a 
public  school  not  designated  high-minority  of  similar  size  in  the  same  PSU.  These  greater  probabilities  of 
selection  were  used  to  ensure  adequate  samples  of  nonpublic-school  students  in  order  to  allow  the 
derivation  of  reliable  estimates  for  such  students.  No  subgroups  (high  minority  schools  or  nonpublic 
schools)  were  oversampled  in  the  long-term  trend  samples. 

The  total  number  of  schools  selected  for  each  age  class  in  both  the  long-term  trend  and  main 
samples  was  such  that  the  predesignated  student  sample  sizes  would  be  achieved  by  selecting  all  eligible 
students  in  a selected  school,  up  to  the  maximum  sizes  indicated  above.  The  target  sample  size  also 
allowed  for  losses  due  to  nonparticipation  of  selected  schools  and  students  and  the  exclusion  of  students 
from  the  assessment.  This  design,  with  the  important  exceptions  described  above,  had  the  goal  of 
yielding  a sample  of  students  in  a given  age  class  or  grade  with  approximately  uniform  probabilities  of 
selection.  The  efforts  to  oversample  nonpublic-school  students  and  minority  students  in  the  main  samples 
and  the  practical  constraints  on  the  sample  size  within  each  school  resulted  in  some  substantial  violations 
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of  this  general  goal.  The  distributions  of  selection  probabilities  of  the  selected  students,  as  reflected  in 
their  sampling  weights,  are  mentioned  in  Chapter  10. 

The  QED  files  do  not  contain  schools  that  opened  between  1994  and  the  time  of  the  assessments. 
Therefore,  special  procedures  were  implemented  to  be  sure  that  the  NAEP  assessment  represented 
students  in  new  public  schools.  Small  school  districts,  which  generally  contained  only  one  eligible  school 
for  a given  age  class,  were  handled  differently  from  large  school  districts,  which  generally  contained 
more  than  one  eligible  school  for  a given  age  class.  In  small  school  districts,  the  schools  selected  for  a 
given  age  class  were  thought  to  contain  all  students  in  the  district  that  were  eligible  for  the  assessment. 
Districts  containing  such  schools  in  the  school  sample  were  asked  if  other  schools  with  the  appropriate 
grades  for  the  assessment  existed,  and  if  so,  they  were  automatically  included  in  the  assessment.  For 
large  school  districts,  a district-level  sampling  frame  was  constructed  from  the  schools  on  the  QED  file 
that  were  eligible  for  one  of  the  national  assessments.  Then  districts  were  sampled  systematically  with 
probabilities  proportional  to  a measure  of  size.  In  most  cases,  the  measure  of  size  was  total  district 
enrollment,  but  a minimum  measure  of  size  was  used  in  districts  below  a certain  cutoff.  Each  sampled 
district  was  asked  to  update  lists  of  eligible  schools  according  to  information  on  the  QED  files.  Sampling 
frames  of  eligible  new  schools  for  these  large  districts  were  then  constructed  separately  for  both  main 
and  long-term  trend  samples  at  each  age  class,  and  separate  samples  of  new  schools  were  selected 
systematically  with  probability  proportional  to  eligible  enrollment  using  the  same  sampling  rates  as  for 
the  original  sample.  Seven  new  schools  were  added  to  the  main  samples:  two  at  grade  4,  three  at  grade  8, 
and  two  at  grade  12.  Four  new  schools  were  added  to  the  long-term  trend  samples:  one  at  age  class  9, 
three  at  age  class  13,  and  none  at  age  class  17.  Although  new  school  sampling  procedures  were  applied  at 
age  class  17,  no  new  schools  were  selected  since  schools  with  the  necessary  characteristics  were  not 
available.  All  new  schools  added  to  the  sample  were  obtained  from  large  districts. 

In  a few  PSUs  where  school  refusals  were  relatively  heavy  for  a particular  sample,  substitute 
school  selections  were  made,  replacing  the  refusals  (to  the  extent  feasible)  with  schools  from  within  the 
same  PSU  and  similar  in  size,  affiliation  (public.  Catholic,  or  other  nonpublic),  grade  span,  and  minority 
composition.  The  goal  of  this  procedure  was  to  maintain  the  student  sample  sizes  needed,  while  keeping 
variance  and  nonresponse  bias  at  acceptable  levels.  For  the  main  samples,  31  substitute  schools  were 
selected  using  this  procedure  (10  at  grades  4 and  8,  and  1 1 at  grade  12),  and  28  substitute  schools  were 
selected  for  the  long-term  trend  samples  (15  at  age  class  9,  7 at  age  class  13,  and  6 at  age  class  17). 

Tables  3-5  and  3-6  show  the  number  of  in-scope  schools  selected,  cooperating,  and  substituted,  in  the 
main  and  long-term  trend  samples,  respectively.  The  participation  rates  given  are  based  on  the  original 
sample  of  schools  (excluding  substitutes).  School  participation  rates  for  grade  8 and  nonpublic  schools  in 
the  main  samples  appear  lower  compared  to  those  achieved  in  1994,  while  the  rates  for  public  schools 
appear  higher.  The  other  response  rates  are  comparable  for  the  two  years.  Note  that  since  the  response 
rates  quoted  do  not  include  the  substitute  selections,  the  potential  for  nonresponse  bias  is  likely  to  be  a 
little  less  than  these  rates  would  indicate.  This  is  because  the  substitute  selections  were  chosen  based  on 
their  similarity  to  the  initially  refusing  selections. 

For  the  main  samples,  the  schools  that  were  participating  with  no  eligible  students  left  for  testing 
had  all  of  their  eligible  students  tested  for  State  NAEP,  so  that  no  students  were  left  for  the  main 
samples.  These  schools  were  accounted  for  by  treating  them  as  nonrespondents  for  weighting.  For  long- 
term trend  at  age  class  13,  the  considerable  numbers  of  schools  selected  with  no  eligible  students 
enrolled  resulted  primarily  from  the  fact  that  some  schools  with  grades  6,  7,  or  9,  but  no  grade  8,  were 
sampled.  Such  schools  had  a reasonable  chance  of  containing  some  age  13  students.  Often  they  did  have 
a number  of  eligible  students,  but  sometimes  they  had  none.  Because  of  the  grade  structure  of  schools, 
this  occurred  most  often  for  age  class  13. 
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Table  3-5 

School  Sample  Sizes,  Refusals,  and  Substitutes  for  the  Main  Samples^ 


Status 

Grade  4 

Grade  8 

Grade  12 

Total 

Public^ 

Nonpublic^ 

Selected,  in  scope 

723 

761 

779 

2,263 

1,392 

871 

Refusals 

99 

127 

160 

386 

212 

174 

Participation  rate  of 
originally  selected  schools 

86% 

83% 

79% 

83% 

85% 

80% 

1994  participation  rate 

86% 

86% 

79% 

83% 

82% 

85% 

Participating,  no  eligible 
students  enrolled 

0 

0 

1 

1 

0 

1 

Participating,  no  eligible 
students  left  for  testing'^ 

20 

42 

27 

89 

30 

59 

Substitutes  participating 

1 

1 

2 

4 

4 

0 

Final  assessed  sample 

605 

593 

593 

1,791 

1,154 

637 

^ The  numbers  in  this  table  reflect  the  full  samples,  consisting  of  schools  in  the  SI,  S2,  and  S3  samples.  These  sample  types  are 
described  in  Section  3.4. 

^ Public,  BI A,  and  DoDEA  schools 
^Catholic  and  other  nonpublic  schools 

^ No  students  were  left  for  testing  because  they  had  been  tested  in  the  State  Assessment 


Table  3-6 

School  Sample  Sizes,  Refusals,  and  Substitutes  for  the  Long-Term  Trend  Samples^ 


Status 

Age  Class  9 

Age  Class  13 

Age  Class  17 

Total 

Selected,  in  scope 

291 

316 

237 

844 

Refusals 

43 

51 

44 

138 

Participation  rate  of  originally  selected 
schools 

85% 

84% 

81% 

84% 

1994  participation  rate 

87% 

82% 

81% 

83% 

Participating,  no  eligible  students  enrolled 

8 

27 

2 

37 

Substitutes  participating 

8 

4 

0 

12 

Final  assessed  sample 

248 

242 

191 

681 

^ The  numbers  in  this  table  reflect  the  full  samples,  consisting  of  schools  in  the  SI , S2,  and  S3  samples.  These  sample  types  are 
described  in  Section  3.4. 


3.4  ASSIGNMENT  OF  SAMPLE  TYPE  TO  SCHOOLS 

In  order  to  determine  the  effect  of  using  different  criteria  for  including  students  with  disabilities 
and  limited  English  proficient  students  in  the  assessment,  three  different  sample  types  were  assigned  to 
the  schools  selected  for  the  main  assessment.  In  sample  type  1 (SI)  schools,  the  inclusion  criteria  for  the 
main  samples  were  identical  to  those  used  in  1990  and  1992  and  were  intended  to  be  somewhat  more 
rigorously  defined  than  those  used  in  the  long-term  trend  samples.  In  sample  type  2 (S2)  schools,  new 
1996  inclusion  criteria  were  used.  In  sample  type  3 (S3)  schools,  the  new  1996  inclusion  criteria  were 
used  and  accommodations  were  offered  to  students  with  disabilities  (SD)  and  students  of  limited  English 
proficiency  (LEP).  For  more  details  of  the  inclusion  criteria  and  their  implementation,  and  the 
accommodations  offered  students,  see  Chapter  5.  The  information  in  this  chapter  and  in  Chapter  5 
applies  to  all  three  sample  types  or  subsamples. 
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Sample  type  was  assigned  to  schools  separately  for  each  grade.  For  schools  that  were  not  also 
selected  for  the  State  Assessment  program,  sample  type  was  assigned  as  follows.  At  grade  4,  20  percent 
of  the  schools  were  assigned  sample  type  1,  45  percent  were  assigned  sample  type  2,  and  35  percent  were 
assigned  sample  type  3.  At  grade  8,  one-sixth  of  the  schools  were  assigned  sample  type  1,  and  five- 
twelfths  each  were  assigned  sample  types  2 and  3.  At  grade  12,  two-thirteenths  of  the  schools  were 
assigned  sample  type  1,  six-thirteenths  were  assigned  sample  type  2,  and  five-thirteenths  were  assigned 
sample  type  3.  Sample  type  was  assigned  so  that  a variety  of  schools  with  respect  to  region,  school  type, 
urbanization,  and  size  were  in  each  sample  type  at  each  grade. 

For  schools  selected  for  both  the  main  samples  and  State  Assessment  program,  sample  type  was 
initially  assigned  as  described  above,  and  then  reassigned  for  the  main  samples  as  follows.  Schools 
retained  their  initial  sample  type  assignment  for  the  State  Assessment.  For  the  national  assessment, 
schools  were  ultimately  assigned  the  same  sample  type  as  for  the  State  Assessment,  with  one  exception. 
Schools  that  were  initially  assigned  to  sample  type  3 for  the  national  assessment  and  sample  type  2 for 
the  State  Assessment,  retained  these  different  respective  sample  types  for  each  assessment.  For  all  other 
schools,  the  sample  type  for  the  main  samples  was  switched  to  match  the  state  sample  type.  The  effect  of 
this  procedure  was  to  assign  sample  type  1 to  somewhat  more  schools  and  sample  types  2 and  3 to 
somewhat  fewer  schools  at  grades  4 and  8 than  initially  assigned. 


3.5  ASSIGNMENT  OF  SESSIONS  TO  SCHOOLS 

Sessions  were  assigned  to  the  selected  schools  found  to  be  in-scope  at  the  time  of  session 
assignment  in  the  following  manner.  First,  the  number  of  sessions  per  school  was  established  (three 
sessions  per  school  were  specified  for  the  long-term  trend  samples  and  for  sample  type  1 for  the  main 
samples.  Five  sessions  per  school  were  specified  for  sample  types  2 and  3 for  the  main  samples  at  grades 
4 and  8,  and  six  sessions  per  school  were  specified  for  sample  types  2 and  3 for  the  main  samples  at 
grade  12).  This  was  the  maximum  number  of  sessions  that  could  be  administered  without  creating  unduly 
small  session  sizes  with  few  eligible  students.  Thus,  in  most  long-term  trend  schools,  for  example,  three 
sessions  were  conducted.  However,  schools  with  fewer  than  20  eligible  students  were  asked  to  conduct 
only  a single  session. 

Session  types  associated  with  each  sample  are  listed  in  Table  3-7.  In  the  main  samples,  four  to 
six  different  session  types  were  conducted  at  each  grade  (mathematics,  science,  mathematics  estimation, 
and  mathematics  theme  at  all  grades;  advanced  mathematics  at  grades  8 and  12;  and  advanced  science  at 
grade  12).  All  of  the  session  types  were  not  offered  for  all  samples.  For  long-term  trend,  four  session 
types  were  conducted  at  age  classes  9 and  13  (spiral  plus  three  tape  sessions),  and  three  session  types 
were  conducted  at  age  class  17  (spiral  plus  two  tape  sessions).  Schools  could  be  assigned  multiple 
sessions  of  the  same  type  (for  example,  two  spiral  and  one  tape  sessions  in  long-term  trend,  or  three 
mathematics  and  three  science  sessions  in  the  main  samples). 

Sessions  were  assigned  to  schools  with  three  aims  in  mind.  The  first  was  to  distribute  students  to 
the  different  session  types  across  the  whole  sample  for  each  age  class  so  that  the  target  numbers  of 
assessed  students  would  be  achieved  (in  each  sample  type  separately  in  the  main  assessments).  The 
second  was  to  maximize  the  number  of  different  session  types  that  were  administered  within  a given 
selected  school,  without  violating  the  minimum  session  sizes  discussed  above.  The  third  was  to  give  each 
student  an  equal  chance  of  being  selected  for  a given  session  type  regardless  of  the  number  of  sessions 
conducted  in  the  school. 
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3.6  SAMPLING  STUDENTS 


To  facilitate  the  sampling  of  students,  a consolidated  list  was  prepared  for  each  school  of  all 
grade-eligible  and  age-eligible  students  (for  long-term  trend)  or  all  grade-eligible  students  (for  the  main 
assessments)  for  the  age  class  for  which  the  school  was  selected.  A systematic  selection  of  eligible 
students  was  made  from  this  list  (unless  all  students  were  to  be  assessed)  to  provide  the  target  sample 
size.  For  schools  assigned  more  than  a single  session  type  (the  vast  majority),  students  were  assigned  by 
Westat  district  supervisors  to  one  of  the  various  session  types  using  specified  procedures. 

For  each  age  class,  separately  for  the  long-term  trend  and  main  samples,  maxima  were 
established  as  to  the  number  of  students  who  would  be  selected  for  a given  school.  In  those  schools  that, 
according  to  information  on  the  sampling  frame,  had  fewer  eligible  students  than  the  established  maxima, 
each  eligible  student  enrolled  at  the  school  was  selected  in  the  sample  for  one  of  the  sessions  assigned  to 
the  school.  In  other  schools,  a sample  of  students  was  drawn,  and  then  students  were  assigned  to  sessions 
as  appropriate.  For  the  main  samples,  the  maximum  sample  sizes  were  established  by  sample  type  in 
terms  of  the  number  of  grade-eligible  students:  72  at  grades  4 and  8,  and  90  at  grade  12  for  sample  type 
1;  120  at  grade  4,  160  at  grade  8,  and  180  at  grade  12  for  sample  types  2 and  3.  For  the  long-term  trend 
samples,  the  maximum  at  each  age  class  was  60  age-eligible  students  (about  80  grade-  plus  age-eligible 
students  in  most  schools).  Note  that  the  number  of  students  actually  selected  for  assessment  in  a long- 
term trend  sample  school  generally  fell  somewhat  below  80,  because  students  who  were  selected  for  one 
of  the  long-term  trend  tape-administered  sessions  and  were  in  the  modal  grade  but  not  age-eligible  were 
subsequently  dropped  from  the  sample.  Similarly,  in  the  main  assessments,  at  grades  8 and  12  in  sample 
types  2 and  3,  students  selected  for  the  advanced  mathematics  and  advanced  science  assessments,  who 
were  subsequently  found  to  be  ineligible  on  the  basis  of  their  courses  taken,  were  dropped  from  the 
samples.  This  reduced  the  sample  size  somewhat  in  these  schools. 

The  sample  of  students  to  be  selected  in  each  school  was  derived  in  the  following  manner,  both 
for  main  and  for  long-term  trend  samples.  On  the  basis  of  data  obtained  from  the  school  characteristics 
and  policies  questionnaire  (or  the  sample  frame  when  the  questionnaire  data  were  not  obtained  in  time) 
an  estimate  of  the  number  of  eligible  students  was  established  for  each  school.  For  the  main  samples,  the 
estimated  number  of  grade-eligible  students  was  used;  for  the  long-term  trend  samples,  the  number  of 
age-eligible  students  was  used.  A Session  Assignment  Form  was  generated  for  each  school,  showing  the 
line  numbers  (described  below)  of  the  students  to  be  selected,  indicating  the  type  of  session  to  be  taken 
by  each  such  student.  These  line  numbers  were  generated  using  a sampling  interval  designed  to  give  the 
appropriate  sample  size  for  each  school.  Thus,  the  overall  sampling  interval  was  1.0  for  schools  in  which 
all  eligible  students  were  to  be  assessed.  The  appropriate  sampling  interval  was  specified  for  schools 
with  larger  numbers  of  eligible  students,  such  as  to  give  the  appropriate  maximum  sample  size  (described 
above  for  each  age  class)  in  the  case  that  the  school  had  an  enrollment  of  eligible  students  exactly  equal 
to  that  predicted. 

If  the  Westat  supervisor  found  that,  when  applied  to  the  numbered  list  of  eligible  students 
assembled  in  the  field  for  each  school,  the  line  numbers  generated  gave  rise  to  a sample  in  excess  of  120 
percent  of  the  appropriate  maximum  sample  size  limit  specified  above,  he  or  she  called  Westat’s  central 
office.  By  use  of  a personal  computer,  new  line  numbers  based  on  the  actual  number  of  eligible  students 
were  generated  and  relayed  to  the  supervisor.  A similar  revision  to  the  line  numbers  was  made  in  the  case 
of  a school  with  a sampling  interval  in  excess  of  1.0,  and  eligible  enrollment  less  than  80  percent  of  that 
initially  estimated.  In  this  latter  case,  the  sample  size  was  increased  to  the  appropriate  level.  This 
procedure  gave  a suitable  compromise  between  control  over  the  sampling  rate  within  each  school  and 
operational  autonomy  and  flexibility  for  Westat  field  supervisors.  Note  that  in  all  cases,  sampling 
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intervals  were  generated  in  Westat’s  central  office,  and  stored  for  use  in  sample  weighting.  Supervisors 
were  not  required  to  derive  or  record  within-school  sampling  rates. 

Table  3-7  shows  the  number  of  students  per  school  who  were  assessed  for  each  assessment.  Note 
that,  for  the  various  print  samples,  the  number  of  students  assessed  per  item  per  school  is  quite  low,  even 
though  typically  dozens  of  students  were  assessed  in  total  in  a particular  school.  Thus,  the  extent  of 
clustering  of  the  sample  is  in  general  quite  modest,  because  most  sampled  schools  conducted  a few 
different  assessments  with  a moderate  number  of  students  in  each,  and  more  importantly  because  the  use 
of  BIB-spiraling  in  the  print-administered  sessions  greatly  alleviated  the  effects  of  clustering  the  samples 
of  students  within  schools,  for  item-level  data. 


Table  3-7 

Number  of  Students  Per  School  for  Each  Session  Type' 


Sample 

Session  Type 

Number 

of 

Assessed 

Students 

Number 

of 

Schools 

Mean 
Number  of 
Students  Per 
Assessment 
Per  School 

Mean 
Number  of 
Students  Per 
Item  Per 
School 

Age  Class  9 

Print  Booklets  51-56 

5,019 

215 

23.3 

3.9 -7.8^ 

Long-Term 

Tape  Booklet  91 

1,852 

127 

14.6 

14.6 

Trend 

Tape  Booklet  92 

1,721 

116 

14.8 

14.8 

Tape  Booklet  93 

1,840 

125 

14.7 

14.7 

Grade  4 

Print  Mathematics 

10,830 

445 

24.3 

5.6 

Main 

Print  Science 

11,578 

421 

27.5 

4.5  - 7.4^ 

Tape  Mathematics  Estimation 

2,115 

120 

17.6 

17.6 

Print  Mathematics  Theme 

4,004 

230 

17.4 

8.7 

Age  Class  1 3 

Print  Booklets  51-56 

5,493 

221 

24.9 

4.1  -8.3^ 

Long-Term 

Tape  Booklet  91 

1,928 

128 

15.1 

15.1 

Trend 

Tape  Booklet  92 

1,866 

125 

14.9 

14.9 

Tape  Booklet  93 

1,864 

124 

15.0 

15.0 

Grade  8 

Print  Mathematics 

11,521 

411 

28.0 

6.5 

Main 

Print  Science 

11,971 

346 

34.6 

5.6 -9.4^ 

Tape  Mathematics  Estimation 

2,244 

104 

21.6 

21.6 

Print  Mathematics  Theme 

4,227 

175 

24.2 

12.1 

Print  Advanced  Mathematics 

2,365 

253 

9.3 

9.3 

Age  Class  17 

Print  Booklets  51-56 

4,669 

186 

25.1 

4.2  - 8.4^ 

Long-Term 

Tape  Booklet  84 

1,848 

133 

13.9 

13.9 

Trend 

Tape  Booklet  85 

1,691 

122 

13.9 

13.9 

Grade  12 

Print  Mathematics 

10,660 

430 

24.8 

5.7 

Main 

Print  Science 

11,481 

401 

28.6 

4.6 -7.7^ 

Tape  Mathematics  Estimation 

1,889 

96 

19.7 

19.7 

Print  Mathematics  Theme 

3,860 

196 

19.7 

9.8 

Print  Advanced  Mathematics 

2,965 

207 

14.3 

14.3 

Print  Advanced  Science 

2,431 

222 

11.0 

11.0 

' The  numbers  in  this  table  reflect  the  full  samples,  consisting  of  schools  in  the  S 1 , S2,  and  S3  samples.  These  sample  types  are  described 
in  Section  3.4. 

^This  number  varied  because  some  item  blocks  appeared  more  than  others  in  the  set  of  booklets  used  for  this  sample. 
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3.7  OVERSAMPLING  OF  SD/LEP  STUDENTS  FOR  MAIN  SAMPLE 
MATHEMATICS  AND  SCIENCE  ASSESSMENTS 

As  noted  earlier,  in  the  main  assessments  for  mathematics  and  science,  the  procedures  for 
assessing  SD  and  LEP  students  varied  by  sample  type.  The  inclusion  procedure  used  in  sample  type  1 
differed  somewhat  from  that  used  in  sample  types  2 and  3.  SD/LEP  students  in  sample  type  3 were 
offered  accommodations  not  available  to  other  students  or  to  SD/LEP  students  in  other  sample  types. 

As  a measure  to  ensure  an  adequate  sample  size  of  SD/LEP  students  from  both  sample  types  2 
and  3,  oversampling  procedures  were  applied  for  SD/LEP  students  at  all  three  grades,  in  sample  types  2 
and  3 for  mathematics,  and  in  sample  type  3 for  science.  In  this  way,  comparisons  of  the  effect  of 
offering  accommodations  to  students  would  have  enhanced  power  to  detect  effects. 

The  procedure  for  carrying  out  the  oversampling  was  somewhat  different  for  grade  4 than  for 
grades  8 and  12.  This  was  because  of  the  presence  of  the  advanced  mathematics  and  advanced  science 
samples  at  grades  8 and  12  only,  which  offered  an  opportunity  to  oversample  in  a way  not  possible  at 
grade  4.  The  general  intent  of  the  oversampling  was,  within  each  school  assigned  sessions  of  regular 
mathematics  in  sample  types  2 and  3,  and  regular  science  in  sample  type  3,  to  select  SD/LEP  students  for 
these  assessments  at  twice  the  rate  at  which  non  SD/LEP  students  were  sampled  (or  to  include  all 
SD/LEP  students  if  there  were  not  sufficient  numbers  to  permit  sampling  at  twice  the  rate).  There  was  no 
oversampling  of  schools  as  part  of  the  procedure. 

At  grade  4,  the  procedure  was  as  follows.  In  each  school  where  oversampling  was  to  occur,  the 
initial  desired  sample  of  students  was  drawn  for  each  session  assigned,  from  the  full  list  of  eligible 
students.  Among  those  students  not  selected  for  any  sessions  in  this  way,  the  SD/LEP  students  were 
identified.  A sample  from  among  these  was  drawn,  using  a sampling  rate  that  would  achieve  the  double 
sampling  rate  required  overall.  In  most  cases,  this  involved  selecting  all  such  SD/LEP  students  in  the 
school.  If  the  school  was  a sample  type  3 school  assigned  to  assess  both  mathematics  and  science,  the 
extra  SD/LEP  students  so  selected  were  split  among  mathematics  and  science  in  the  same  proportion  as 
the  initial  student  sample  for  the  school.  Thus,  if  the  school  was  assigned  two  sessions  of  science  and  one 
of  mathematics,  two-thirds  of  these  extra  SD/LEP  students  were  assigned  to  science,  and  one-third  to 
mathematics. 

The  sampling  of  additional  SD/LEP  students  was  carried  out  using  designated  line  numbers, 
indicated  on  the  session  assignment  form  used  to  generate  the  samples  of  students  in  each  school.  In  this 
way,  the  necessary  information  as  to  the  selection  probability  of  each  student  was  retained  for  use  in 
weighting.  No  reliance  was  placed  on  information  generated  in  the  field.  Field  supervisors  had  only  to 
follow  the  prespecified  sampling  instructions. 

At  grades  8 and  12,  a different  approach  was  taken.  As  a result  of  the  pattern  of  assigning 
sessions  to  schools,  it  was  the  case  that  in  every  school  in  which  there  were  students  remaining  who  were 
not  selected  for  assessment  in  the  initial  sampling  phase  (so  that  there  were  in  fact  SD/LEP  students 
available  for  oversampling),  a session  of  either  advanced  mathematics,  or  advanced  science  (at  grade  12), 
was  assigned.  This  was  the  result  of  the  scheme  for  assigning  sessions  to  schools  efficiently;  it  was  not  a 
condition  imposed  in  order  to  facilitate  oversampling.  The  SD/LEP  students  assigned  as  an  oversample 
for  the  regular  mathematics  and  (in  the  case  of  sample  type  3)  science  assessments  were  those  SD/LEP 
students  who  were  initially  selected  for  the  advanced  mathematics  and  science  samples,  but  who  were  not 
eligible  for  those  assessments  because  they  had  not  taken  the  appropriate  set  of  courses.  Thus,  for  grades 
8 and  12,  the  oversampling  of  SD/LEP  students  took  place  only  among  that  subpopulation  that  was  not 
eligible  for  the  advanced  assessments. 
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It  was  assumed  that  there  would  be  relatively  few  SD/LEP  students  who  would  qualify  for  the 
advanced  sessions,  since  nationally  about  20  percent  of  all  students  were  so  eligible.  To  the  extent  to 
which  there  were  SD/LEP  students  who  qualified  for  the  advanced  mathematics  and  science  assessments, 
however,  the  oversampling  procedure  for  regular  mathematics  and  science  was  not  biased,  because  this 
was  taken  into  account  in  the  weighting  of  the  regular  mathematics  and  science  assessments.  This  was 
possible  because,  for  the  students  in  the  regular  mathematics  and  science  samples,  those  who  were 
SD/LEP  were  identified,  and  those  who  qualified  for  the  advanced  assessments  were  also  identified. 

All  such  additional  SD/LEP  students  were  included  unless  this  would  have  led  to  the  sampling 
rate  of  SD/LEP  students  within  the  school  being  more  than  twice  the  rate  of  other  students.  In  such  cases, 
a random  subsample  of  the  extra  SD/LEP  students  was  selected.  As  for  grade  4,  all  information  needed  in 
the  field  to  carry  out  the  oversampling  was  contained  on  the  preprogrammed  Session  Assignment  Form, 
so  that  the  complex  weighting  process  could  be  carried  without  the  possibility  of  error  being  introduced 
in  sampling  information  obtained  from  the  field.  Also,  as  in  grade  4,  in  sample  type  3 schools  that  were 
assigned  both  regular  mathematics  and  regular  science  sessions,  the  extra  SD/LEP  students  sampled  were 
assigned  in  the  appropriate  proportions. 

Since  the  aim  was  to  oversample  by  a factor  of  two  where  possible,  the  overall  rate  of 
oversampling  of  SD/LEP  students  was  instead  less  than  two.  That  is  because  in  smaller  schools  there 
were  no  students  remaining  who  had  not  already  been  assigned  to  a session.  Again,  the  weighting 
procedures  ensured  that  the  results  were  not  biased  as  a result  of  the  relative  under  representation  of 
SD/LEP  students  from  smaller  schools. 

Table  3-8  shows  the  results  of  the  oversampling  efforts  for  each  grade  and  sample  type  for 
mathematics  and  science.  The  weighted  results  show  the  proportion  of  the  sample  that  would  have  been 
SD/LEP  students  had  no  oversampling  been  attempted.  The  focus  should  be  on  sample  types  2 and  3 for 
mathematics  and  sample  type  3 for  science,  since  this  is  where  the  oversampling  of  SD/LEP  students 
occurred.  The  extent  to  which  the  unweighted  percentage  of  SD/LEP  students  exceeds  the  weighted 
percentage  is  a measure  of  the  effectiveness  of  the  oversampling.  As  can  be  seen,  the  procedure  was 
effective  in  increasing  the  sample  of  SD/LEP  students  considerably  at  grades  8 and  12  for  both  subjects, 
but  was  not  very  effective  at  grade  4 for  either  subject.  To  have  increased  the  sample  of  SD/LEP  students 
further  at  grade  4 would  have  required  the  assessment  of  additional  schools. 


Table  3-8 

Percentage  of  Sampled  Students  Who  Were  Specified  as  SD/LEP 
in  the  1996  Main  Samples  - Mathematics  and  Science 

Subject/  Grade  4 Grade  8 Grade  12 

Sample  Type’  Unweighted  Weighted  Unweighted  Weighted  Unweighted  Weighted 


Mathematics/S  1 

12.4% 

13.2% 

9.5% 

10.2% 

6.4% 

6.8% 

Mathematics/S2 

17.9% 

15.7% 

19.9% 

11.5% 

15.3% 

8.1% 

Mathematics/S3 

17.0% 

15.4% 

18.9% 

12.1% 

15.7% 

8.5% 

Total 

15.8% 

14.7% 

16.1% 

11.3% 

12.7% 

7.8% 

Science/S2 

16.5% 

15.8% 

13.1% 

11.9% 

9.5% 

9.2% 

Science/S3 

17.7% 

16.7% 

18.9% 

10.9% 

13.6% 

7.4% 

Total 

16.9% 

16.2% 

15.2% 

11.4% 

10.9% 

8.3% 

' The  numbers  in  this  table  reflect  the  full  samples,  consisting  of  schools  in  the  SI , S2,  and  S3  samples.  These  sample  types  are 
described  in  Section  3.4. 
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3.8  EXCLUDED  STUDENTS 


School  staff  completed  an  SD/LEP  questionnaire  for  each  sampled  student  identified  as  lEP 
(with  an  individualized  education  program)  or  LEP.  Some  of  these  students  were  deemed  unassessable  by 
school  authorities  and  were  excluded  from  the  assessment.  For  the  long-term  trend  samples,  a distinct 
sample  of  excluded  students  was  identified  at  each  age  class.  For  the  main  samples,  a distinct  sample  of 
excluded  students  was  identified  for  each  subject  and  sample  type  combination. 

The  inclusion  criteria  for  the  main  samples  differed  somewhat  from  those  used  for  the  long-term 
trend  samples.  In  addition,  the  inclusion  criteria  for  the  main  samples  differed  by  sample  type.  In  sample 
type  1 schools,  the  inclusion  criteria  for  the  main  samples  were  identical  to  those  used  in  1990  and  1992, 
and  were  intended  to  be  somewhat  more  rigorously  defined  than  those  used  in  the  long-term  trend 
samples.  In  sample  type  2 schools,  new  1996  inclusion  criteria  were  used.  In  sample  type  3 schools,  the 
new  1996  inclusion  criteria  were  used  and  accommodations  were  offered  to  SD/LEP  students. 

For  the  long-term  trend  samples,  the  inclusion  criteria  were  the  same  as  in  past  long-term  trend 
assessments,  dating  back  to  the  early  1980s. 

For  all  samples,  students  were  selected  for  specific  sessions,  and.the  school  was  then  asked  to 
identify  those  to  be  excluded.  Thus,  only  age-eligible  students  were  considered  for  inclusion  in  the  long- 
term trend  tape-administered  sessions,  whereas  both  age-  and  grade-eligible  students  were  considered  for 
inclusion  in  the  print-administered  long-term  trend  samples.  The  samples  of  excluded  students  for  the 
long-term  trend  samples  were  weighted  in  such  a way  as  to  account  for  this  procedure  appropriately  (see 
Chapter  10). 


Table  3-9 

Student  Exclusion  Rates  by  Age  Class  and  School  Type  and  Sample  Type,  Weighted 


Subject/ 
Sample  Type’ 

Public 

Age  Class  9 
Non- 
Public 

Total 

Age  Class  13 
Non- 

Public  Public 

Total 

Age  Class  17 
Non- 

Public  Public 

Total 

Main  Samples 

Mathematics/S  1 

5.6% 

1.0% 

5.2% 

4.5% 

0.2% 

4.1% 

3.4% 

0.1% 

3.0% 

Mathematics/S2 

9.1% 

1.3% 

8.1% 

4.7% 

0.1% 

4.3% 

3.5% 

0.1% 

3.2% 

Mathematics/S3 

4.4% 

0.0% 

3.9% 

3.4% 

0.0% 

3.1% 

3.1% 

0.2%  • 

2.8% 

Science/S2 

9.2% 

0.3% 

8.2% 

4.7% 

0.2% 

4.3% 

4.3% 

0.4% 

0.2% 

3.9% 

Science/S3 

6.5% 

0.1% 

5.9% 

3.7% 

0.3% 

3.4% 

2.8% 

2.6% 

Estimation/All 

6.7% 

0.0% 

5.8% 

5.1% 

0.0% 

4.7% 

4.0% 

0.0% 

3.5% 

Theme/ All 

7.6% 

0.6% 

6.8% 

4.4% 

0.0% 

4.0% 

3.6% 

1.4% 

3.4% 

Advanced  Mathematics/All 

— 

— 

■ — 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

Advanced  Science/ All 

— 

— 

— 

— 

— 

— 

0.0% 

0.0% 

0.0% 

Long-Term  Trend  Samples 

Reading/Writing  Print 

6.6% 

0.3% 

5.9% 

5.9% 

0.4% 

5.4% 

5.3% 

0.2% 

5.0% 

Mathematics/Science  Tape 

6.3% 

0.6% 

5.6% 

4.8% 

0.2% 

4.3% 

3.5% 

0.4% 

3.3% 

* The  numbers  in  this  table  reflect  the  full  samples,  consisting  of  schools  in  the  S I , S2,  and  S3  samples.  These  sample  types  are  described  in 
Section  3.4. 


o 

ERIC 


96 


69 


Table  3-9  shows  the  rates  of  exclusion  for  each  age  class,  subject,  sample  type  (for  math  and 
science),  and  school  type  for  the  long-term  trend  and  main  samples.  The  most  marked  effects  are  the 
much  higher  rates  of  exclusion  in  public  schools  than  in  nonpublic,  and  the  higher  rates  of  exclusion  at 
lower  grades.  The  former  phenomenon  is  no  doubt  a function  of  the  greater  prevalence  of  special 
education  and  language  minority  programs  in  public  schools.  The  higher  exclusion  rates  at  lower  ages, 
which  occurred  also  in  other  years,  result  from  the  greater  proportion  of  students  at  these  grades  who  are 
excluded  for  reasons  of  limited  English  proficiency.  In  certain  areas  of  the  United  States,  fourth-grade 
public-school  students  whose  native  language  is  Spanish  are  taught  predominantly  in  Spanish,  and  in 
these  schools  a very  high  proportion  of  sampled  students  are  excluded.  Factors  that  may  limit  the 
comparability  of  these  rates  to  those  in  previous  years  are  the  different  inclusion  criteria,  oversampling  of 
SD/LEP  in  some  subjects,  the  different  subjects  assessed,  and  the  inclusion  of  only  grade  eligible 
students  in  the  main  samples  in  1996. 

3.9  STUDENT  PARTICIPATION  RATES 

Table  3-10  summarizes  the  rates  of  participation  of  invited  students.  The  set  of  invited  students 
consists  of  the  selected  students,  after  removing  the  excluded  students  and,  in  the  case  of  long-term  trend 
samples,  removing  those  students  selected  for  tape-administered  sessions  who  were  not  age-eligible.  For 
a given  session,  a makeup  session  was  called  for  when,  for  various  reasons,  more  than  a predetermined 
tolerable  number  of  invited  students  failed  to  attend  the  originally  scheduled  session  to  which  they  were 
invited.  The  participation  rates  given  in  the  table  express  the  number  finally  assessed  as  a percentage  of 
those  initially  invited  in  the  participating  schools.  Participation  rates  are  shown  for  the  main  and  long- 
term trend  samples  and  for  public  and  nonpublic  schools  separately  in  the  case  of  the  main  samples. 
Overall  participation  rates  are  also  shown  for  comparable  samples  from  the  1994  NAEP  assessment.  The 
table  shows  that  student  participation  rates  in  1996  are  similar  to  those  experienced  in  1994.  The  rates 
increased  slightly  at  age  class  9 for  both  samples,  and  remained  fairly  steady  for  the  other  samples.  At  all 
age  classes,  the  participation  rate  of  nonpublic-school  students  exceeds  that  of  public-school  students, 
with  the  difference,  both  relative  and  absolute,  increasing  with  age  class. 


Table  3-10 

Student  Participation  Rates  by  Age  Class  and  School  Type,  Unweighted' 


1996  Public 

1996  Nonpublic 

1996  Combined 

1994 

Number 

Participation 

Number 

Participation 

Number 

Participation 

Participation 

Samples 

Invited 

Rate 

Invited 

Rate 

Invited 

Rate^ 

Rate^ 

\ge  Class  9 

Long-Term  Trend 

9,715 

95.4% 

1,204 

96.8% 

10,919 

95.5% 

94.2% 

Main 

24,082 

95.1% 

5,834 

96.6% 

29,916 

95.4% 

93.2% 

Age  Class  13 

Long-Term  Trend 

10,980 

91.6% 

1,152 

95.0% 

12,132 

91.9% 

92.2% 

Main 

28,351 

92.3% 

6,368 

96.7% 

34,719 

93.1% 

91.0% 

4ge  Class  17 

Long-Term  Trend 

9,051 

83.4% 

717 

91.9% 

9,768 

84.0% 

84.1% 

Main 

34,199 

77.3% 

7,473 

91.5% 

41,672 

79.9% 

81.1% 

‘ The  numbers  in  this  table  reflect  the  full  samples,  including  all  sample  types  (see  Section  3.4). 

^ Somewhat  different  inclusion  criteria  were  used  for  the  main  samples  than  for  the  long-term  trend  samples  in  each  year,  and  for  the  main 


samples  in  1 994  versus  1 996.  The  total  rates  for  the  main  samples  are  based  on  a relatively  greater  contribution  from  nonpublic-school  students. 
Nonpublic-school  students  constitute  about  18%  of  the  1996  main  samples,  16%  of  the  1994  main  samples,  and  1 1%  of  the  1994  and  1996 
long-term  trend  samples. 


3.10  OVERALL  STUDENT  PARTICIPATION  RATES 


The  combined  impact  of  school  nonparticipation  and  student  absenteeism  from  sessions  within 
participating  schools  is  summarized  in  Table  3-11.  The  table  shows  the  percentages  of  students  assessed, 
from  among  those  who  would  have  been  assessed  if  all  initially  selected  schools  had  participated,  and  if 
all  invited  students  had  attended  either  an  initial  or  make-up  session.  The  results  show  that,  consistent 
with  earlier  rounds  of  NAEP,  the  overall  level  of  participation  decreases  substantially  with  the  increase 
in  age  and  grade  of  the  students. 

So  far  in  this  chapter,  only  unweighted  participation  rates  by  age  class  and  school  type  have  been 
presented.  However,  analysis  is  typically  performed  separately  by  age  class  and  session  type,  and  NCES 
standards  regarding  acceptable  potentials  for  bias  are  expressed  in  terms  of  weighted  participation  rates. 
Therefore,  Tables  3-12  and  3-13  show  weighted  participation  rates  by  age  class  and  session  type  for  the 
main  and  long-term  trend  samples,  respectively.  The  main  sample  rates  are  for  students  in  the  reporting 
populations.  Note  that  for  the  main  samples,  the  student  participation  rates  are  similar  for  different 
session  types  at  grades  4 and  8,  but  the  student  participation  rates  at  grade  12  and  the  school  participation 
rates  at  all  grades  vary  by  session  type.  The  differential  school  participation  rates  reflect  the  fact  that, 
more  so  than  in  previous  years,  different  session  types  include  different  schools.  This  is  due  to  the 
assignment  of  schools  to  sample  type,  the  fact  that  all  session  types  were  not  assessed  in  all  sample  types, 
and  the  specific  sample  types  included  in  the  reporting  populations  for  each  session  type  (see  Chapter 
10).  For  long-term  trend,  the  participation  rates  are  similar  for  different  session  types  in  the  same  age 
class.  They  are  also  similar,  in  general,  to  the  unweighted  rates. 

The  procedures  for  substituting  for  nonparticipating  schools  or  imputing  for  them  through 
weighting  and  the  procedures  for  imputing  for  absent  students  through  weighting  were  designed  (so  far 
as  feasible)  to  reduce  the  biases  resulting  from  school  and  student  nonparticipation.  These  procedures  are 
discussed  in  Chapter  10. 


Table  3-11 

Overall  Participation  Rates  (School  and  Student  Combined)  by  Age  Class,  Unweighted^ 


1996  Samples 

Age  Class  9 

Age  Class  13 

Age  Class  17 

Overall 

Main  Samples 

School  participation 
Student  participation 
Overall  student  participation 

Number  of  participating  students 

86.3% 

95.4% 

82.3% 

28,527 

83.3% 

91.5% 

76.2% 

32,328 

79.5% 

79.9% 

63.5% 

33,286 

82.9% 

88.6% 

73.4% 

94,141 

Long-Term  Trend  Samples 

School  participation 

85.2% 

83.9% 

81.4% 

83.6% 

Student  participation 

95.5% 

91.9% 

84.0% 

90.8% 

Overall  student  participation 

81.4% 

77.1% 

68.4% 

75.9% 

Number  of  participating  students 

10,432 

11,151 

8,208 

29,791 

Overall 

School  participation 

86.0% 

83.5% 

79.9% 

83.1% 

Student  participation 

95.4% 

92.8% 

80.7% 

89.1% 

Overall  student  participation 

82.0% 

77.5% 

64.5% 

74.0% 

Number  of  participating  students 

38,959 

43,479 

41,494 

123,932 

* The  numbers  in  this  table  reflect  the  full  samples,  including  all  sample  types  (see  Section  3.4). 
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Table  3-12 

Weighted  Participation  Rates  by  Age  Class  and  Session  Type,  1996  Main  NAEP  Reporting  Samples 


Participation 
(Sample  Type) 

Mathematics 

Print 

Science 

Print 

Mathematics 

Estimation 

Print 

Mathematics 
Theme  Print 

Advanced  , 
Mathematics 
Print 

Advanced 

Science 

Print 

Grade  4 

School  participation 

82.3% 

77.8% 

93.5% 

77.9% 

— 

— 

Student  participation 

95.3% 

94.9% 

96.7% 

95.4% 



Overall  participation 

78.4% 

73.8% 

90.4% 

74.4% 

Grade  8 

School  participation 

81.5% 

79.7% 

85.3% 

86.8% 

77.0% 

— 

Student  participation 

92.9% 

93.1% 

93.8% 

92.7% 

95.6% 

Overall  participation 

75.7% 

74.3% 

80.0% 

80.4% 

73.6% 

Grade  12 

School  participation 

76.2% 

77.4% 

63.9% 

78.4% 

77.6% 

77.7% 

Student  participation 

82.3% 

77.5% 

81.0% 

78.2% 

85.8% 

86.5% 

Overall  participation 

62.7% 

60.0% 

51.7% 

61.3% 

66.6% 

67.2% 

Table  3-13 

Weighted  Participation  Rates  by  Age  Class  and  Session  Type 
1996  Long-Term  Trend  Samples 


Participation 

ReadingAV  riting 
Print 

Mathematics/Science 

Tape 

Age  Class  9 

School  participation 

83.5% 

82.6% 

Student  participation 

95.6% 

95.4% 

Overall  participation 

79.9% 

78.8% 

Age  Class  13 

School  participation 

82.0% 

80.8% 

Student  participation 

92.2% 

92.6% 

Overall  participation 

75.6% 

74.8% 

Age  Class  17 

School  participation 

81.7% 

75.6% 

Student  participation 

83.8% 

84.1% 

Overall  participation 

68.5% 

63.6% 

3.11  SAMPLING  TEACHERS 

The  teacher  questionnaire  was  administered  to  teachers  of  fourth-  and  eighth-grade  students 
assessed  in  mathematics  and  science  and  twelfth-grade  students  assessed  in  advanced  mathematics. 
Teachers  were  given  the  questionnaire  if  they  taught  the  student  the  subject  in  which  the  student  was 
assessed.  The  purpose  of  drawing  these  samples  was  not  to  estimate  the  attributes  of  the  teacher 
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population,  but  to  estimate  the  number  (proportion)  of. students  whose  teachers  had  various  attributes  and 
to  correlate  student  characteristics  and  performance  with  the  characteristics  of  their  teachers. 

The  selected  teachers  were  asked  to  complete  a questionnaire  concerning  themselves  and  their 
teaching  practices,  with  specific  references  to  each  individual  class  period  containing  a student  included 
in  the  main  assessment. 
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Chapter  4 


ASSESSMENT  INSTRUMENTS^ 

Stephen  Lazer 
Educational  Testing  Service 


4.1  INTRODUCTION 

In  the  1996  assessment,  four  types  of  instruments  vyere  used  to  collect  data  about  students, 
teachers,  and  schools.  Each  assessed  student  received  an  assessment  booklet  containing  both  cognitive 
and  background  questions.  An  SD/LEP  Student  Questionnaire  was  completed  by  school  officials  for 
each  sampled  student  identified  as  having  a disability  (SD)  or  classified  as  Limited  English  Proficient 
(LEP),  whether  or  not  the  students  were  able  to  participate  in  the  assessment.  The  teachers  of  fourth-, 
eighth-,  and  twelfth-grade  students  participating  in  the  assessment  were  asked  to  complete  a Teacher 
Questionnaire.  A School  Characteristics  and  Policies  Questionnaire  was  distributed  to  each 
participating  school. 

This  chapter  begins  with  a discussion  of  the  characteristics  of  the  student  booklets  used  for  the 
1996  main  and  long-term  trend  assessments  and  how  the  booklets  were  assembled.  The  contents  of  each 
booklet  and  item  block  is  presented  in  detail  in  a set  of  tables.  Section  4.4  describes  the  student,  teacher, 
SD/LEP,  and  school  questionnaires  that  were  part  of  the  1996  assessment. 


4.2  STUDENT  BOOKLETS— MAIN  ASSESSMENTS 

4.2.1  Mathematics 

Each  student  assessed  in  mathematics  received  a booklet  containing  a set  of  general  background 
questions,  content  questions,  subject-specific  background  questions,  and  questions  about  his  or  her 
motivation  and  familiarity  with  the  assessment  materials.  The  passages  and  content  questions  were 
assembled  into  sections  or  blocks.  Students  in  the  main  assessment  were  given  three  15-minute  blocks. 
Those  sampled  for  the  theme  assessment  completed  one  15-minute  block  and  one  30-minute  block.  Those 
sampled  for  the  advanced  study  at  grade  8 completed  three  20-minute  blocks;  at  grade  12  advanced 
sample  students  completed  three  30-minute  blocks.  Students  in  the  estimation  sample  completed  one  15- 
minute  block  from  the  main  assessment  and  two  paced-tape  sections.  The  overall  assessment  time  for 
each  student  was  approximately  63  minutes. 

The  assembly  of  blocks  into  booklets  for  the  main  assessment  and  their  subsequent  assignment  to 
sampled  students  was  determined  by  a balanced  incomplete  block  (BIB)  design  with  spiraled 
administration.  The  student  booklets  contained  two  five-minute  background  sections,  a one-minute 
background  section,  and  three  15-minute  blocks  of  items  according  to  a BEB  design. 

The  BEB  design  for  the  1996  national  mathematics  assessment  focused  by  subject  area,  so 
that  students  received  booklets  containing  only  blocks  of  mathematics  questions  (not  science).  The  BEB 
design  also  balances  the  order  of  presentation  of  the  15-minute  blocks  of  items — every  15-minute  block 

' Stephen  Lazer  manages  assessment  development  activities  for  the  NAEP  program  at  ETS. 

101 


75 


appears  as  the  first  cognitive  block  in  two  booklets,  as  the  second  cognitive  block  in  two  other  booklets, 
and  as  the  third  cognitive  block  in  another  two  booklets. 

The  design  used  in  1996  required  that  13  blocks  of  mathematics  items  at  each  grade  be  assembled 
into  26  booklets.  Theme  blocks  vvere  placed  in  two  other  booklets,  and  estimation  blocks  in  one  other 
booklet.  At  grades  8 and  12,  the  advanced  study  was  placed  in  one  additional  booklet.^  Once  assembled, 
the  main  assessment  booklets  were  then  spiraled  and  bundled.  Spiraling  involves  interweaving  the 
booklets  in  a systematic  sequence  so  that  each  booklet  appears  an  appropriate  number  of  times  in  the 
sample.  The  bundles  were  designed  so  that  each  booklet  would  appear  equally  often  in  each  position  in  a 
bundle. 


The  final  step  in  the  BIB-spiraling  procedure  was  the  assigning  of  the  booklets  to  the  assessed 
students.  The  students  within  an  assessment  session  were  assigned  booklets  in  the  order  in  which  the 
booklets  were  bundled.  Thus,  most  students  in  an  assessment  session  received  different  booklets.  In  the 
assessment  design,  representative  and  randomly  equivalent  samples  of  students  responded  to  each  item  at 
a given  grade  level. 

Tables  4-1,  4-2,  and  4-3  provide  the  composition  and  number  of  booklets  administered  in  the 
1996  mathematics  assessment.  Table  4-4  gives  details  of  the  item  blocks  used  in  the  main  assessment, 
including  the  number  of  cognitive  items  in  each  block  and  the  booklets  in  which  each  block  appeared; 
Table  4-5  gives  the  same  information  for  blocks  in  the  special  components  of  the  assessment.  Table  4-6 
gives  pertinent  information  about  the  background  sections. 


^ See  Chapter  2 for  descriptions  of  these  types  of  assesment  blocks. 


Table  4-1 

Main  Sample  Booklet  Configuration 
Grade  4,  Mathematics 


Booklet 

Number 

Background 

Common  Mathematics 

Cognitive 

Blocks 

Motivation 
Background  Block 

101 

C13 

M2 

M3 

M4 

M7 

MX 

102 

C13 

M2 

M4 

M5 

,M8 

MX 

103 

C13 

M2 

M5 

M6 

M9 

MX 

104 

C13 

M2 

M6 

M7 

MIO 

MX 

105 

C13 

M2 

M7 

M8 

Mil 

MX 

106 

C13 

M2 

M8 

M9 

M12 

MX 

107 

C13 

M2 

M9 

MIO 

M13 

MX 

108 

C13 

M2 

MIO 

Mil 

M14 

MX 

109 

C13 

M2 

Mil 

M12 

M15 

■ MX 

110 

C13 

M2 

M12 

M13 

M3 

MX 

111 

C13 

M2 

M13 

M14 

M4 

MX 

112 

C13 

M2 

M14 

M15 

M5 

MX 

113 

C13 

M2 

M15 

M3 

M6 

MX 

114 

C13 

M2 

M3 

M5 

MIO 

MX 

115' 

C13 

M2 

M4 

M6 

Mil 

MX 

116 

C13 

M2 

M5 

M7 

M12 

, MX 

117 

C13 

M2 

M6 

M8 

M13 

MX 

118 

C13 

M2 

M7 

M9 

M14 

MX 

119 

C13 

M2 

M8 

MIO 

M15 

MX 

120 

C13 

M2 

M9 

Mil 

M3 

MX 

121^ 

C13 

M2 

MIO 

M12 

M4 

MX 

122 

C13 

M2 

Mil 

M13 

M5 

MX 

123 

C13 

M2 

M12 

M14 

M6 

MX 

124 

C13 

M2 

M13 

M15 

M7 

MX 

125 

C13 

M2 

M14 

M3 

M8 

MX 

126 

C13 

M2 

M15 

M4 

M9 

MX 

127^ 

C13 

M2 

M4 

M16 

M17 

MX 

128“ 

C13 

M2 

M4 

M21 

— 

MX 

129“ 

C13 

M2 

M4 

M22 

— 

MX 

921^ 

C13 

M2 

MIO 

M12 

M4 

MX 

^ This  booklet  was  a large  print  version. 

^This  booklet  was  also  used  for  SD/LEP  students  who  took  a regular-print  version. 

^This  was  an  estimation  booklet  (involved  paced  audiotapes). 

'‘This  was  a theme  booklet. 

^This  was  a bilingual  booklet  presented  to  some  SD/LEP  students.  It  contained  the  same  blocks  as  Booklet  Number  121. 
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Table  4-2 

Main  Sample  Booklet  Configuration 
Grade  S,  Mathematics 


Booklet 

Number 

Background 

Common  Mathematics 

Cognitive 

Blocks 

Motivation 
Background  Block 

101 

C13 

M2 

M3 

M4 

M7 

MX 

102 

C13 

M2 

M4 

M5 

M8 

MX 

103 

C13 

M2 

M5 

M6 

M9 

MX 

104 

C13 

M2 

M6 

M7 

MIO 

MX 

105 

C13 

M2 

M7 

M8 

Mil 

MX 

106 

C13 

M2 

M8 

M9 

M12 

MX 

107 

C13 

M2 

M9 

MIO 

M13 

MX 

108 

C13 

M2 

MIO 

Mil 

M14 

MX 

109 

C13 

M2 

Mil 

M12 

M15 

MX 

no 

C13 

M2 

M12 

M13 

M3 

MX 

111 

C13 

M2 

M13 

M14 

M4 

MX 

112 

C13 

M2 

M14 

M15 

M5 

MX 

113 

C13 

M2 

M15 

M3 

M6 

MX 

114 

C13 

M2 

M3. 

M5 

MIO 

MX 

115' 

C13 

M2 

M4 

M6 

Mil 

MX 

116 

C13 

M2 

M5 

M7 

M12 

MX 

117 

C13 

M2 

M6 

M8 

M13 

MX 

118 

C13 

M2 

M7 

M9 

M14 

MX 

119 

C13 

M2  . 

M8 

MIO 

M15 

MX 

120 

C13 

M2 

M9 

Mil 

M3 

MX 

121^ 

C13 

M2 

MIO 

M12 

M4 

MX 

122 

C13 

M2 

Mil 

M13 

M5 

MX 

123 

C13 

M2 

M12 

M14 

M6 

MX 

124 

C13 

M2 

M13 

M15 

M7 

MX 

125 

C13 

M2 

M14 

M3 

M8 

MX 

126 

C13 

M2 

M15 

M4 

M9 

MX 

127" 

C13 

M2 

M4 

M16 

M17 

MX 

128“ 

C13 

M2 

M4 

M21 



MX 

129“ 

C13 

M2 

M4 

M22 

— 

MX 

130" 

C13 

M2 

M20 

M18 

M19 

MX 

9215/6 

C13 

M2 

MIO 

M12 

M4 

MX 

' This  booklet  was  a large  print  version. 

^This  booklet  was  also  used  for  SD/LEP  students  who  took  a regular-print  version. 

^This  was  an  estimation  booklet  (involved  paced  audiotapes). 

"*This  was  a theme  booklet. 

^ This  was  an  advanced  booklet. 

^This  was  a bilingual  booklet  presented  to  some  SD/LEP  students.  It  contained  the  same  blocks  as  Booklet  Number  121 . 
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Table  4-3 

Main  Sample  Booklet  Configuration 
Grade  12,  Mathematics 


Booklet 

Number 

Background 

Common  Mathematics 

Cognitive 

Blocks 

Motivation 
Background  Block 

101 

C13 

M2 

M3 

M4 

M7 

MX 

102 

C13 

M2 

M4 

M5 

M8 

MX 

103 

C13 

M2 

M5 

M6 

M9 

MX 

104 

C13 

M2 

M6 

M7 

MIO 

MX 

105 

C13 

M2 

M7 

M8 

Mil 

MX 

106 

C13 

M2 

M8 

M9 

M12 

MX 

. 107 

C13 

M2 

M9 

MIO 

M13 

MX 

108 

C13 

M2 

MIO 

Mil 

M14 

MX 

109 

C13 

M2 

Mil 

M12 

M15 

MX 

no 

C13 

M2 

M12 

M13 

M3 

MX 

111 

C13 

M2 

M13 

M14 

M4 

MX 

112 

C13 

M2 

M14 

M15 

M5 

MX 

113 

C13 

M2 

M15 

M3 

M6 

MX 

114 

C13 

M2 

M3 

M5 

MIO 

MX 

115' 

C13 

M2 

M4 

M6 

Mil 

MX 

116 

C13 

M2 

M5 

M7 

M12 

MX 

117 

C13 

M2 

M6 

M8 

M13 

MX 

118 

C13 

M2 

M7 

M9 

M14 

MX 

119 

C13 

M2 

M8 

MIO 

M15 

MX 

120 

C13 

M2 

M9 

Mil 

M3 

MX 

121^ 

C13 

M2 

MIO 

M12 

M4 

MX 

122 

C13 

M2 

Mil 

M13 

M5 

MX 

123 

C13 

M2 

M12 

M14 

M6 

MX 

124 

C13 

M2 

M13 

M15 

M7 

MX 

125 

C13 

M2 

M14 

M3 

M8 

MX 

126 

C13 

M2 

M15 

M4 

M9 

MX 

\2t 

C13 

M2 

M4 

M16 

M17 

MX 

128^ 

C13 

M2 

M4 

M21 

— • ■ 

MX 

129'' 

C13 

M2 

M4 

M22 

— 

MX 

130^ 

C13 

M2 

M20 

M18 

M19 

MX 

‘ This  booklet  was  a large  print  version. 

^This  booklet  was  used  for  SD/LEP  students  who  took  a regular-print  version. 
^This  was  an  estimation  booklet  (involved  paced  audiotapes). 

^ This  was  a theme  booklet. 

^ This  was  an  advanced  booklet. 
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Table  4-4 

1996  Mathematics  Assessment,  Main  BIB 


Block 

Designation 

Grade 

Multiple- 

Choice 

Short 

Constructed- 

Response 

Extended 

Constructed- 

Response 

Total 

3 

S1M3 

4 

9 

4 

0 

13 

S2M3 

8 

9 

3 

1 

13 

S3M3 

12 

10 

4 

0 

14 

4 

S123M4a 

4 

14 

0 

0 

14 

S123M4b 

8 

21 

0 

0 

21 

S123M4C 

12 

22 

0 

0 

22 

5, 

S12M5a 

4 

4 

5 

1 

10 

S12M5b 

8 

6 

4 

1 

11 

S3M5 

12 

4 

5 

1 

10 

6 

S123M6a 

4 

0 

11 

0 

11 

S123M6b 

8 

0 

16 

0 

16 

S123M6C 

12 

0 

17 

0 

17 

7 

S12M7a 

4 

3 

4 

1 

8 

S12M7b 

8 

5 

4 

1 

10 

S3M7 

12 

4 

5 

1 

10 

8 

S123M8a 

4 

14 

1 

0 

15 

S123M8b 

8 

16 

2 

0 

18 

S123M8C 

12 

17 

4 

0 

21 

9 

S1M9 

4 

9 

2 

1 

12 

S23M9b 

8 

5 

3 

1 

9 

S23M9C 

12 

6 

2 

1 

9 

10 

S123M10a 

4 

0 

6 

0 

6 

S123M10b 

8 

0 

7 

0 

7 

S123M10C 

12 

3 

6 

1 

10 

11 

S12Mlla 

' 4 

11 

5' 

0 

16 

S12Mllb 

8 

13 

6 

0 

19 

S3M11 

12 

11 

3 

0 

14 

12 

S1M12 

4 

5 

3 

1 

9 

S23M12b 

8 

4 

4 

1 

9 

S23M12C 

12 

4 

5 

1 

10 

13 

S1M13 

4 

6 

5 

1 

12 

S2M13 

8 

6 

4 

1 

11 

S3M13 

12 

3 

5 

1 

9 

14 

S1M14 

4 

4 

5 

1 

10 

S23M14b 

8 

5 

3 

1 

9 

S23M14C 

12 

5 

3 

1 

9 

15 

S1M15 

4 

3 

6 

1 

10 

S2M15 

8 

4 

4 

1 

9 

S3M15 

12 

4 

5 

1 

10 

Comments 

Trend  (92) 

Trend  (92) 

Trend  (92) 

Trend  (90) 

Trend  (90) 

Trend  (90) 

New 

New 

New  --  Calc 
Trend  (90) 

Trend  (90) 

Trend  (90) 

New  “ Manip 
New  ~ Manip 
New  “ Manip 
Trend  Calc  (90) 

Trend  Calc  (90) 

Trend  Calc  (90) 

Trend  (92) 

Trend  (92) 

Trend  (92) 

Manipulatives  (92) 
Manipulatives  (92) 
Manipulatives  (92) 

Trend  (92) 

Trend  (92) 

Trend  (92) 

New  Calculator 
New  Calculator 
New  Calculator 
Trend  (92) 

Trend  (92) 

Trend  Prot  (92) 

New  Calculator 
New  Calculator 
New  Calculator 
New  --  Ruler 

New  — Calculator/Protractor 
New  “ Calculator 
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Table  4-5 

1996  Mathematics  Assessment,  Estimation  and  Targeted  Assessment 


Block 

Designation 

Grade 

Multiple- 

Choice 

Short 

Constructed- 

Response 

Extended 

Constructed- 

Response 

Total 

Comments 

16 

S123M16a 

4 

20 

0 

0 

20 

Trend  Estimation  (90) 

S123M16b 

8 

22 

0 

0 

22 

Trend  Estimation  (90) 

S123M16C 

12 

22 

0 

0 

22 

Trend  Estimation  (90) 

17 

S1M17 

4 

10 

3 

0 

13 

New  Estimation 

S2M17 

8 

9 

6 

0 

15 

New  Estimation 

S3M17 

12 

16 

0 

0 

16 

New  Estimation 

18 

S2M18 

8 

3 

6 

1 

10 

New  Algebra 

S3M18 

12 

3 

6 

2 

11 

New  Advanced  Mathematics 

19 

S2M19 

8 

6 

5 

1 

12 

New  Algebra 

S3M19 

12 

4 

5 

2 

11 

New  Advanced  Mathematics 

21' 

S1M21 

4 

1 

4 

3 

8 

New  Theme 

S23M21b 

8 

4 

5 

2 

11 

New  Theme 

S23M21C 

12 

4 

5 

2 

11 

New  Theme 

22 

S1M22 

4 

0 

3 

3 

6 

New  Theme 

S2M22 

8 

4 

4 

2 

10 

New  Theme 

S2M23 

12 

4 

2 

1 

7 

New  Theme 

‘ Block  20  was  a block  composed  of  exercises  from  the  main  assessment  used  for  linking. 


Table  4-6 

Background  Sections  of  Student  Mathematics  Booklets 


Number  of  Questions 

Placement  in  Student  Booklet 

Grade  4 

General  Background 

24 

Section  1 

Mathematics  Background 

25 

Section  2 

Motivation 

5 

Section  6^ 

Grade  8 

General  Background 

26 

Section  1 

Mathematics  Background 

31 

Section  2 

Motivation 

5 

Section  6* 

Grade  12 

General  Background 

35 

Section  1 

Mathematics  Background 

44 

Section  2 

Motivation 

5 

Section  6 ' 

* Or  Section  5 in  theme  booklets. 
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4.2.2  Science 


Each  student  assessed  in  science  received  a booklet  containing  general  background  questions, 
content  questions,  subject-specific  background  questions,  and  questions  about  his  or  her  motivation  and 
familiarity  with  the  assessment  materials.  The  passages  and  content  questions  were  assembled  into 
sections  or  blocks.  Students  in  the  main  assessment  were  given  three  20-minute  blocks  at  grade  4,  and 
three  30-minute  blocks  at  grades  8 and  12.  The  last  block  in  every  book  was  a hands-on  block.  Those 
sampled  for  the  advanced  study  at  grade  12  completed  four  30-minute  blocks.  The  overall  assessment 
time  for  each  student  was,  on  average,  120  minutes. 

The  assembly  of  blocks  into  booklets  for  the  main  assessment  and  their  subsequent  assignment  to 
sampled  students  was  determined  by  a complex  design  with  spiraled  administration.  The  student  booklets 
contained  two  five-minute  background  sections,  a one-minute  background  section,  and  three  blocks  of 
items. 


The  design  for  the  1996  national  assessment  was  focused  by  subject  area,  so  that  students 
received  booklets  containing  only  blocks  of  science  questions  (not  mathematics).  The  design  also 
balances  the  order  of  presentation  of  the  blocks  of  items,  except  for  the  hands-on  blocks,  which  always 
appear  in  position  three  of  a booklet.  All  other  blocks  appear  an  equal  number  of  times  in  position  one 
and  position  two.  Further,  the  design  was  set  up  to  ensure  that  no  student  answered  more  than  one  theme- 
based  block  (though  some  students  did  not  receive  any).  This  design  allows  for  some  balancing  of  the 
impact  of  context  and  fatigue  effects  to  be  measured  and  reported,  but  makes  allowance  for  the 
difficulties  and  disruption  of  administering  hands-on  blocks.  It  also  takes  into  account  the  limited  breadth 
of  content  coverage  included  in  the  theme  blocks. 

The  design  used  in  1996  required  that  15  blocks  of  science  items  at  each  grade  be  assembled  into 
37  booklets.  At  grade  12,  the  advanced  study  was  composed  of  three  additional  booklets.  Once 
assembled,  the  main  assessment  booklets  were  then  spiraled  and  bundled.  Spiraling  involves 
interweaving  the  booklets  in  a systematic  sequence  so  that  each  booklet  appears  an  appropriate  number 
of  times  in  the  sample.  The  bundles  were  designed  so  that  each  booklet  would  appear  equally  often  in 
each  position  in  a bundle. 

The  final  step  in  the  spiraling  procedure  was  the  assigning  of  the  booklets  to  the  assessed 
students.  The  students  within  an  assessment  session  were  assigned  booklets  in  the  order  in  which  the 
booklets  were  bundled.  Thus,  most  students  in  an  assessment  session  received  different  booklets.  In  the 
assessment  design,  representative  and  randomly  equivalent  samples  of  about  1,200  students  responded  to 
each  item  at  a given  grade  level. 

Tables  4-7,  4-8,  and  4-9  provide  the  composition  and  number  of  booklets  administered  in  the 
main  1996  science  assessment.  Table  4-10  provides  the  composition  of  booklets  in  the  advanced  science 
study.  Table  4-1 1 gives  details  of  the  item  blocks  used  in  the  main  assessment,  including  the  number  of 
cognitive  items  in  each  block  and  the  booklets  in  which  each  block  appeared;  Table  4-12  gives  the  same 
information  for  blocks  in  the  special  components  of  the  assessment.  Table  4-13  gives  pertinent 
information  about  the  background  sections. 


Table  4-7 

Main  Sample  Booklet  Configuration 
Grade  4,  Science 


Booklet 

Number 

Cognitive 

Blocks 

Hands-On 

Task' 

Background 
Common  Science 

Motivation 
Background  Block 

201 

S7 

SIO 

S3 

C19 

S2 

SX 

202 

S7 

Sll 

S4 

C19 

S2 

sx 

203 

S7 

S12 

S5 

C19 

S2 

SX 

204 

S7 

S13 

S6 

C19 

S2 

sx 

205 

SIO 

Sll 

S3 

C19 

S2 

sx 

206 

S12 

S8 

S4 

C19 

S2 

sx 

207 

SIO 

S13 

S5 

C19 

S2 

sx 

208 

SIO 

S8 

S6 

C19 

S2 

sx 

209 

Sll 

S12 

S3 

C19 

S2 

sx 

210 

S13 

S14 

S4 

C19 

S2 

sx 

211 

Sll 

S8 

S5 

C19 

S2 

sx 

212 

Sll 

S14 

S6 

C19 

S2 

sx 

213 

S13 

S8 

S3 

C19 

S2 

sx 

214 

S8 

S15 

S4 

C19 

S2 

sx 

215 

S12 

S14 

S5 

C19 

S2 

sx 

216 

S12 

S15 

S6 

C19 

S2 

sx 

217 

S8 

S14 

S3 

C19 

S2 

sx 

218 

S14 

S20 

S4 

C19 

S2 

sx 

219 

S8 

S20 

S5 

C19 

S2 

sx 

220^ 

S13 

S20 

S6 

C19 

S2 

sx 

221 

S14 

S15 

S3 

C19 

S2 

sx 

222 

S15 

S21 

S4 

C19 

S2 

sx 

223 

S15 

S9 

S5 

C19 

S2 

sx 

224 

S8 

S21 

S6 

C19 

S2 

sx 

225 

S20 

S21 

S3 

C19 

S2 

sx 

226 

S20 

S9 

S4 

C19 

S2 

sx 

227 

S20 

S7 

S5 

C19 

S2 

sx 

228 

S14 

S9 

S6 

C19 

S2 

sx 

229 

S21 

S9 

S3 

C19 

S2 

sx 

230 

S21 

S7 

S4 

C19 

S2 

sx 

231 

S21 

SIO 

S5 

C19 

S2 

sx 

232 

S15 

S7 

S6 

C19 

S2 

sx 

233 

S9 

S13 

S3 

C19 

S2 

sx 

234 

S9 

SIO 

S4 

C19 

S2 

sx 

235 

S9 

Sll 

S5 

C19 

S2 

sx 

236 

S9 

S12 

S6 

C19 

S2 

sx 

237 

S14 

S7 

S3 

C19 

S2 

sx 

’ Hands-on  task  blocks:  Block  S3  uses  “A"  kit  - seeds;  Block  S4  uses  “B”  kit  - unknown  powders;  Block  S5  uses  “C”  kit  - 
floating  pencil;  and  Block  S6  uses  “D’*  kit  - markers. 

^This  booklet  was  also  used  for  SD/LEP  students  who  took  a regular-print  version. 
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Table  4-8 

Main  Sample  Booklet  Configuration 
Grade  8,  Science 


Booklet 

Number 

Cognitive 

Blocks 

Hands-On 

Task* 

Background 
Common  Science 

Motivation 
Background  Block 

201 

S7 

SIO 

S3 

C19 

S2 

SX 

202 

S7 

Sll 

S4 

C19 

S2 

sx 

203 

S7 

S12 

S5 

C19 

S2 

SX 

204 

S7 

S13 

S6 

C19 

S2 

sx 

205 

SIO 

Sll 

S3 

C19 

S2 

sx 

206 

S12 

S8 

S4 

C19 

S2 

sx 

207 

SIO 

S13 

S5 

C19 

S2 

sx 

208 

SIO 

S8 

S6 

C19 

S2 

sx 

209 

Sll 

S12 

S3 

C19 

S2 

sx 

210 

S13 

S14 

S4 

C19 

S2 

sx 

211 

Sll 

S8 

S5 

C19 

S2 

sx 

212 

Sll 

S14 

S6 

C19 

S2 

sx 

213 

S13 

S8 

S3 

C19 

S2 

sx 

214 

S8 

S15 

S4 

C19 

S2 

sx 

215 

S12 

S14 

S5 

C19 

S2 

sx 

216 

S12 

S15 

S6 

C19 

S2 

sx 

217 

S8 

S14 

S3 

C19 

S2 

sx 

218 

S14 

S20 

S4 

C19 

S2 

sx 

219 

S8 

S20 

S5 

C19 

S2 

sx 

220^ 

S13 

S20 

S6 

C19 

S2 

sx 

221 

S14 

S15 

S3 

C19 

S2 

sx 

222 

S15 

S21 

S4 

C19 

S2 

sx 

223 

S15 

S9 

S5 

C19 

S2 

sx 

224 

S8 

S21 

S6 

C19 

S2 

sx 

225 

S20 

S21 

S3 

C19 

S2 

sx 

226 

S20 

S9 

S4 

C19 

S2 

sx 

227 

S20 

S7 

S5 

C19 

S2 

sx 

228 

S14 

S9 

S6 

C19 

S2 

sx 

229 

S21 

S9 

S3 

C19 

S2 

sx 

230 

S21 

S7 

S4 

C19 

S2 

sx 

231 

S21 

SIO 

S5 

C19 

S2 

sx 

232 

S15 

S7 

S6 

C19 

S2 

sx 

233 

S9 

S13 

S3 

C19 

S2 

sx 

234 

S9 

SIO 

S4 

C19 

S2 

sx 

235 

S9 

Sll 

S5 

C19 

S2 

sx 

236 

S9 

S12 

S6 

C19 

S2 

sx 

237 

S14 

S7 

S3 

C19 

S2 

sx 

' Hands-On  task  blocks:  Block  S6  uses  “D”  kit  - markers;  Block  S3  uses  “E”  kit  - powders;  Block  S4  uses  “F’  kit  - salt 
solutions;  and  Block  S5  uses  “G”  kit  - soil  tests. 

^This  booklet  was  also  used  for  SD/LEP  students  who  took  a regular-print  version. 
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Table  4-9 

Main  Sample  Booklet  Configuration 
Grade  12,  Science 


Booklet 

Number 

Cognitive 

Blocks 

Hands-On 

Task* 

Background 
Common  Science 

Motivation 
Background  Block 

201 

S7 

SIO 

S3 

C19 

S2 

SX 

202 

S7 

Sll 

S4 

C19 

S2 

sx 

203 

S7 

S12 

S5 

C19 

S2 

SX 

204 

S7 

S13 

S6 

C19 

S2 

sx 

205 

SIO 

Sll 

S3 

C19 

S2 

sx 

206 

S12 

S8 

S4 

C19 

S2 

sx 

207 

SIO 

S13 

S5 

C19 

S2 

sx 

208 

SIO 

S8 

S6 

C19 

S2 

sx 

209 

Sll 

S12 

S3 

C19 

S2 

sx 

210^ 

S13 

S14 

S4 

C19 

S2 

sx 

211 

Sll 

S8 

S5 

C19 

S2 

sx 

212 

Sll 

S14. 

S6 

C19 

S2 

sx 

213 

S13 

S8 

S3 

C19 

S2 

sx 

214 

S8 

S15 

S4 

C19 

S2 

sx 

215 

S12 

S14 

S5 

C19 

S2 

sx 

216 

S12 

S15 

S6 

C19 

S2 

sx 

217 

S8 

S14 

S3 

C19 

S2 

sx 

218 

S14 

S20 

S4 

C19 

S2 

sx 

219 

S8 

S20 

S5 

C19 

S2 

sx 

220 

S13 

S20 

S6 

C19 

S2 

sx 

221 

S14 

S15 

S3 

C19 

S2 

sx 

222 

S15 

S21 

S4 

C19 

S2 

sx 

223 

S15 

S9 

S5 

C19 

S2 

sx 

224 

S8 

S21 

S6 

C19 

S2 

sx 

225 

S20 

S21 

S3 

C19 

S2 

sx 

226 

S20 

S9 

S4 

C19 

S2 

sx 

227 

S20 

S7 

S5 

C19 

S2 

sx 

228 

S14 

S9 

S6 

C19 

S2 

sx 

229 

S21 

S9 

S3 

C19 

S2 

sx 

230 

S21 

S7 

S4 

C19 

S2 

sx 

231 

S21 

SIO 

S5 

C19 

S2 

sx 

232 

S15 

S7 

S6 

C19 

S2 

sx 

233 

S9 

S13 

S3 

C19 

S2 

sx 

234 

S9 

SIO 

S4 

C19 

S2 

sx 

235 

S9 

Sll 

S5 

C19 

S2 

sx 

236 

S9 

S12 

S6 

C19 

S2 

sx 

237 

S14 

S7 

S3 

C19 

S2 

sx 

' Hands-On  task  blocks:  Block  S5  uses  “G”  kit  - soil  tests;  Block  S3  uses  “H”  kit  - antacid;  Block  S4  uses  “I”  kit  - 
separation;  and  Block  S6  uses  “J”  kit  - pendulum. 

^This  booklet  was  also  used  for  SD/LEP  students  who  took  a regular-print  version. 
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Table  4-10 

Main  Sample  Booklet  Configuration 
Grade  12,  Advanced  Science 


Booklet 

Number 

Cognitive 

Blocks 

Hands-On 

Task 

Background 
Common  Science 

Motivation 
Background  Block 

238 

S19 

S18 

S17 

S16 

C19 

S2 

SX 

239 

S19 

S17 

S16 

S18 

C19 

S2 

sx 

240 

S19 

S16 

S18 

S17 

C19 

S2 

SX 

V..  V 


5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

20 

21 


Table  4-11 

1996  Science  Assessment,  Main  BIB 

Short  Extended 
Multiple-  Constructed-  Constructed- 

Designation  Grade  Choice  Response  Response  Total  Comments 


S1S3 

4 

0 

7 

S2S3 

8 

0 

4 

S3S3 

12 

0 

4 

S1S4 

4 

1 

0 

S2S4 

8 

3 

4 

S3S4 

12 

0 

1 

S1S5 

4 

0 

5 

S23S5 

8/12 

0 

6 

S12S6A 

4 

0 

0 

S12S6B 

8 

0 

5 

S3S6 

12 

0 

6 

S1S7 

4 

0 

10 

S2S7 

8 

4 

10 

S3S7 

12 

5 

7 

S1S8 

4 

1 

6 

S23S8A 

8 

5 

5 

S23S8B 

12 

6 

7 

S12S9A 

4 

2 

6 

S12S9B 

8 

3 

9 

S3S9 

12 

4 

8 

SISIO 

4 

6 

4 

S2S10 

8 

8 

7 

S3S10 

12 

7 

7 

SlSll 

4 

6 

5 

S2S11 

8 

8 

7 

S3S11 

12 

7 

5 

S1S12 

4 

6 

4 

S23S12 

8/12 

8 

6 

S1S13 

4 

6 

4 

S23S13 

8/12 

8 

7 

S12S14A 

4 

5 

5 

S12S14B 

8 

7 

11 

S3S14 

12 

8 

6 

S12S15A 

4 

3 

5 

S12S15B 

8 

7 

7 

S3S15 

12 

0 

5 

S1S20 

4 

6 

2 

S2S20 

8 

8 

6 

S3S20 

12 

7 

6 

S1S21’ 

4 

5 

3 

S2S21 

8 

7 

7 

S3S21 

12 

8 

4 

0 

7 

Hands-on 

2 

6 

Hands-on 

2 

6 

Hands-on 

7 

7 

Hands-on 

3 

10 

Hands-on 

3 

4 

Hands-on 

6 

11 

Hands-on 

0 

6 

Hands-on 

4 

4 

Hands-on 

2 

7 

Hands-on 

2 

8 

Hands-on 

0 

10 

Theme-based 

0 

14 

Theme-based 

3 

15 

Theme-based 

1 

8 

Theme-based 

0 

10 

Theme-based 

1 

14 

Theme-based 

1 

9 

Theme-based 

1 

13 

Theme-based 

2 

14 

Theme-based 

1 

11 

Concept/Problem-Solving 

1 

16 

Concept/Problem-Solving 

1 

15 

Concept/Problem-Solving 

0 

11 

Concept/Problem-Solving 

1 

16 

Concept/Problem-Solving 

3 

15 

Concept/Problem-Solving 

1 

10 

Concept/Problem-Solving 

2 

16 

Concept/Problem-Solving 

1 

11 

Concept/Problem-Solving 

1 

16 

Concept/Problem-Solving 

0 

10 

Concept/Problem-Solving 

0 

18 

Concept/Problem-Solving 

2 

16 

Concept/Problem-Solving 

1 

9 

Concept/Problem-Solving 

2 

16 

Concept/Problem-Solving 

2 

7 

In-depth 

3 

11 

Concept/Problem-Solving 

2 

16 

Concept/Problem-Solving 

3 

16 

Concept/Problem-Solving 

2 

10 

Concept/Problem-Solving 

2 

16 

Concept/Problem-Solving 

4 

16 

Concept/Problem-Solving 
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Table  4-12 

1996  Science  Assessment,  Advanced  Assessment 


Block 

Designation 

Grade 

Multiple- 

Choice 

Short 

Constructed- 

Response 

Extended 

Constructed- 

Response 

Total 

Comments 

16 

S3S16 

12 

7 

8 

1 

16 

Advanced  Block 

17 

S3S17 

12 

7 

5 

4 

16 

Advanced  Block 

18 

S3S18 

12 

7 

4 

5 

16 

Advanced  Block 

19 

S3S19 

12 

9 

7 

2 

18 

Advanced  Block 

Table  4-13 

Background  Sections  of  1996  Student  Science  Booklets 


Number  of  Questions 

Placement  in  Student  Booklet 

Grade  4 

General  Background 

24 

Section  4 

Science  Background 

39 

Section  5 

Motivation 

5 

Section  6 

Grade  8 

General  Background 

26 

Section  4 

Science  Background 

42 

Section  5 

Motivation 

5 

Section  6 

Grade  12 

General  Background 

36 

Section  4 

Science  Background 

53 

Section  5 

Motivation 

5 

Section  6 

4.3  STUDENT  BOOKLETS— LONG-TERM  TREND  ASSESSMENTS 

There  were  several  long-term  trend  samples  in  the  1994  assessment  (see  Chapter  1),  each  of 
which  required  the  use  of  special  booklets.  Tables  4-14, 4-15,  and  4-16  summarize  the  contents  of  each 
trend  assessment  booklet  and  show  how  many  of  each  booklet  were  administered.  Tables  4-20,  4-21,  and 
4"22  give  details  of  the  item  blocks  used  in  the  long-term  trend  assessments,  including  the  number  of 
cognitive  and  constructed-response  items  in  each  block  and  the  booklets  in  which  each  block  appeared. 

Reading  and  Writing  Long-Term  Trend.  Six  booklets  (numbered  51  to  56)  containing  reading  and 
writing  items  were  administered  to  each  age  class.  These  booklets  were  identical  to  booklets  used  in 
previous  assessments  of  reading  and  writing  and  were  spiraled  for  administration.  Each  booklet  consisted 
of  a common  background  block  (BZ)  and  three  cognitive  blocks  (at  least  one  reading  block  and  at  least 
one  writing  block).  In  addition  to  cognitive  items,  the  cognitive  blocks  also  contained  subject-related 
background  questions. 

Mathematics  and  Science  Long-Term  Trend.  Three  booklets  (91,  92,  and  93)  at  ages  9 and  13  and 
two  booklets  (84  and  85)  at  age  17,  containing  mathematics  and  science  items,  were  identical  to  those 
used  in  previous  assessments  to  measure  trends.  Each  booklet  contained  a common  background  block 
(Cl  or  BZ)  and  three  cognitive  blocks.  At  ages  9 and  13,  these  booklets  contained  one  reading  block  (Rl, 


R2,  or  R3),  one  mathematics  block  (Ml,  M2,  or  M3)  and  one  science  block  (SI,  S2,  or  S3).  At  age  17, 
each  booklet  contained  at  least  one  mathematics  block  (Ml  to  M3)  and  at  least  one  science  block  (SI  - 
S3).  Mathematics  block  M3  contained  items  that  required  the  use  of  a calculator.  All  cognitive  blocks 
also  contained  subject-related  background  questions. 


Table  4-14 

Long-Term  Trend  Sample  Booklet  Contents  and  Number  of  Booklets  Administered 

Age  Class  9 


Subject 

Area 

Booklet 

Number 

Common 

Background 

Block 

Subject 

Area 

Background 

Block' 

Cognitive  Blocks 

Number  of 
Booklets 
Administered 

Reading  and 

51 

BZ 

— 

BC 

BL 

BQ 

1,186 

Writing 

52 

BZ 

— 

BH 

BE 

BR 

1,165 

53 

BZ 

— 

BC 

BK 

BJ 

1,178 

54 

BZ 

— 

BG 

BO 

BE 

1,180 

55 

BZ 

— 

BM 

BG 

BN 

1,169 

56 

BZ 

— 

BV 

BR 

1,184 

Mathematics 

91 

Cl 

— 

R1 

Ml 

SI 

2,388 

and  Science 

92 

Cl 

— 

S2 

R2 

M3^ 

2,512 

93 

Cl 

— 

M2 

S3 

R3 

2,435 

' Subject  area  background  questions  are  included  in  cognitive  blocks  for  this  booklet. 
^ Calculator  needed  for  this  block. 
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Table  4-15 

Long-Term  Trend  Sample  Booklet  Contents  and  Number  of  Booklets  Administered 

Age  Class  13 


Subject 


Common 

Area 

Number  of 

Subject 

Booklet 

Background 

Background 

Booklets 

Area 

Number 

Block 

Block' 

Cognitive  Blocks 

Administered 

Reading  and 

51 

BZ 

— 

BM 

BK 

BD 

919 

Writing 

52 

BZ 

— 

BC 

BL 

BQ 

906 

53 

BZ 

— 

BH 

BE 

BR 

923 

54 

BZ 

— 

BN 

BC 

BD 

905 

55 

BZ 

— 

BG 

BO 

BE 

928 

56 

BZ 

— 

BG 

BJ 

BP 

933 

Mathematics 

91 

Cl 

— 

R1 

Ml 

SI 

1,928 

and  Science 

92 

Cl 

— 

S2 

R2 

M3^ 

1,976 

93 

Cl 

— 

M2 

S3 

R3 

2,005 

^ Subject  area  background  questions  are  included  in  cognitive  blocks  for  this  booklet. 
^ Calculator  needed  for  this  block. 


Table  4-16 

Long-Term  Trend  Sample  Booklet  Contents  and  Number  of  Booklets  Administered 

Age  Class  17 


Subject 

Area 

Booklet 

Number 

Common 

Background 

Block 

Subject 

Area 

Background 

Block' 

Cognitive  Blocks 

Number  of 
Booklets 
Administered 

Reading  and 

51 

BZ 

— 

BM 

BK 

BD 

927 

Writing 

52 

BZ 

— 

BC 

BL 

BQ 

924 

53 

BZ 

— 

BH 

BE 

BR 

917 

54 

BZ 

— 

BN 

BC 

BD 

951 

55 

BZ 

— 

BG 

BO 

BE 

939 

56 

BZ 

' 

BG 

BJ 

BP 

911 

Mathematics 

84  . 

Cl 



Ml 

M2 

S3 

2,207 

and  Science 

85 

Cl 

— 

SI 

S2 

M3^ 

2,152 

* Subject  area  background  questions  are  included  in  cognitive  blocks  for  this  booklet. 
^ Calculator  needed  for  this  block. 
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Table  4-17 

Long-Term  Trend  Sample  Block  Information,  Age  Class  9 


Block 

Type 

Total 
Number 
of  Items 

Number 

of 

Cognitive 

Items 

Number  of 
Open-Ended  Items 
Cognitive 
Noncognitive 

Booklets 

Containing 

Block 

BZ 

Common  Background 

37 

0 

0 

1 

51-56 

Cl 

Common  Background 

28 

0 

0 

0 

91-93 

BC 

Writing  Background/Cognitive 

23 

1 

1 

0 

51,53 

BE 

Writing  Background/Cognitive 

11 

2 

2 

0 

52,54 

BG 

Writing  Background/Cognitive 

8 

2 

2 

0 

54,  55 

BH 

Reading  Background/Cognitive 

15 

11 

1 

0 

52 

BJ 

Reading  Background/Cognitive 

24 

13 

1 

0 

53 

BK 

Reading  Background/Cognitive 

19 

11 

0 

0 

53 

BL 

Reading  Background/Cognitive 

26 

7 

1 

1 

51 

BM 

Reading  Background/Cognitive 

16 

12 

1 

0 

55 

BN 

Reading  Background/Cognitive 

25 

14 

1 

0 

55 

BO 

Reading  Background/Cognitive 

22 

11 

0 

0 

54 

BQ 

Reading  Background/Cognitive 

21 

12 

0 

0 

51 

BR 

Reading  Background/Cognitive 

16 

12 

0 

0 

52,  56 

BV 

Reading  and  Writing 
Background/Cognitive 

36 

7 Rd. 
1 Wr. 

1 Rd.  0 

1 Wr. 

56 

R1 

Reading  Background/Cognitive 

20 

9 

0 

0 

91 

R2 

Reading  Background/Cognitive 

20 

11 

0 

0 

92 

R3 

Reading  Background/Cognitive 

17 

10 

1 

0 

93 

Ml 

Mathematics 

26 

26 

9 

0 

91 

M2 

Background/Cognitive 

26 

26 

9 

0 

93 

M3 

Mathematics 

Background/Cognitive 

Mathematics 

Background/Cognitive  (Calc.) 

19 

16 

10 

0 

92 

SI 

Science  Background/Cognitive 

23 

18 

0 

0 

91 

S2 

Science  Background/Cognitive 

25 

25 

0 

0 

92 

S3 

Science  Background/Cognitive 

31 

20 

0 

0 

93 

111 
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Table  4-18 

Long-Term  Trend  Sample  Block  Information,  Age  Class  13 


Number  of 


Block 

Type 

Total 
Number 
of  Items 

Number  of 
Cognitive 
Items 

Open-Ended  Items 
Cognitive 
Noncognitive 

Booklets 

Containing 

Block 

BZ 

Common  Background 

37 

0 

0 

1 

51-56 

Cl 

Common  Background 

30 

0 

0 

0 

91-93 

BC. 

Writing  Background/Cognitive 

23 

1 

1 

0 

52,  54 

BD 

Writing  Background/Cognitive 

25 

1 

1 

0 

51,54 

BE 

Writing  Background/Cognitive 

11 

2 

2 

0 

53,  55 

BG 

Writing  Background/Cognitive 

8 

2 

2 

0 

55,56 

BH 

Reading  Background/Cognitive 

18 

13 

1 

1 

53 

BJ 

Reading  Background/Cognitive 

24 

14 

2 

0 

56 

BK 

Reading  Background/Cognitive 

17 

9 

1 

0 

51 

BL 

Reading  Background/Cognitive 

27 

6 

1 

1 

52 

BM 

Reading  Background/Cognitive 

16 

12 

1 

0 

51 

BN 

Reading  Background/Cognitive 

23 

12 

1 

0 

54 

BO 

Reading  Background/Cognitive 

21 

10 

2 

0 

55 

BP 

Reading  Background/Cognitive 

15 

9 

1 

0 

55 

BQ 

Reading  Background/Cognitive 

23 

17 

0 

0 

52 

BR 

Reading  Background/Cognitive 

19 

15 

0 

0 

53 

R1 

Reading  Background/Cognitive 

31 

12 

. 1 

0 

91 

R2 

Reading  Background/Cognitive 

19 

10 

0 

0 

92 

R3 

Reading  Background/Cognitive 

28 

13 

0 

0 

93 

Ml 

Mathematics 

51 

37 

9 

0 

91 

M2 

Background/Cognitive 

44 

37 

8 

0 

93 

M3 

Mathematics 

Background/Cognitive 

Mathematics 

Background/Cognitive  (Calc.) 

32 

24 

10 

0 

92 

SI 

Science  Background/Cognitive 

36 

25 

0 

0 

91 

S2 

Science  Background/Cognitive 

40 

27 

0 

0 

92 

S3 

Science  Background/Cognitive 

36 

27 

0 

0 

93 

O I 

ERIC 


118 


Table  4-19 

Long-Term  Trend  Sample  Block  Information,  Age  Class  17 


Block 

Type 

Total 
Number 
of  Items 

Number  of 
Cognitive 
Items 

Number  of 
Open-Ended  Items 
Cognitive 
Noncognitive 

Booklets 

Containing 

Block 

BZ 

Common  Background 

48 

0 

0 

1 

51-56 

Cl 

Common  Background 

48 

0 

0 

0 

84,  85 

BC 

Writing  Background/Cognitive 

23 

1 

1 

0 

52,  54 

BD 

Writing  Background/Cognitive 

25 

1 

1 

0 

51,54 

BE 

Writing  Background/Cognitive 

11 

2 

2 

0 

53,55 

BG 

Writing  Background/Cognitive 

8 

2 

2 

0 

55,56 

BH 

Reading  Background/Cognitive 

19 

13 

1 

2 

53 

BJ 

Reading  Background/Cognitive 

17 

6 

2 

1 

56 

BK 

Reading  Background/Cognitive 

17 

9 

1 

0 

51 

BL 

Reading  Background/Cognitive 

32 

6 

1 

2 

52 

BM 

Reading  Background/Cognitive 

16 

12 

1 

0 

51 

BN 

Reading  Background/Cognitive 

32 

12 

1 

1 

54 

BO 

Reading  Background/Cognitive 

24 

13 

1 

0 

55 

BP 

Reading  Background/Cognitive 

25 

11 

1 

0 

56 

BQ 

Reading  Background/Cognitive 

17 

11 

1 

0 

52 

BR 

Reading  Background/Cognitive 

20 

9 

0 

0 

53 

Ml 

Mathematics  Background/Cognitive 

49 

35 

10 

0 

84 

M2 

Mathematics  Background/Cognitive 

49 

35 

5 

0 

84 

M3 

Mathematics  Background/Cognitive 
(Calculator) 

35 

24 

14 

0 

85 

SI 

Science  Background/Cognitive 

38 

27 

0 

0 

85 

S2 

Science  Background/Cognitive 

41 

32 

0 

0 

85 

S3 

Science  Background/Cognitive 

32 

23 

0 

0 

84 

4.4  STUDENT,  TEACHER,  AND  SCHOOL  QUESTIONNAIRES 

4.4.1  Student  Questionnaires 

Each  booklet  in  the  main  assessment  included  three  student  background  questionnaires.  The  first, 
consisting  of  general  background  questions,  included  questions  about  race/ethnicity,  mother’s  and 
father’s  level  of  education,  reading  materials  in  the  home,  homework,  attendance,  academic  expectations, 
and  which  parents  lived  at  home.  The  second,  consisting  of  subject-area  background  questions,  included 
questions  about  instructional  activities,  courses  taken,  use  of  specialized  resources  such  as  calculators  in 
mathematics  class,  and  views  on  the  utility  and  value  of  the  subject  matter.  Students  were  given  five 
minutes  to  complete  each  of  these  questionnaires,  with  the  exception  of  the  fourth  graders,  who  were 
given  more  time  because  the  items  in  the  general  questionnaire  were  read  aloud  for  them.  The  third 
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questionnaire  followed  the  three  cognitive  blocks  and  contained  five  questions  about  students’  motivation 
to  do  well  on  the  assessment,  their  perceptions  concerning  the  difficulty  of  the  assessment,  and  their 
familiarity  with  types  of  questions  included. 

The  student  questionnaires  are  described  in  detail  in  Chapter  2. 


4.4.2  Teacher  Questionnaires 

To  supplement  the  information  on  instruction  reported  by  students,  the  mathematics  teachers  of 
the  students  participating  in  the  mathematics  assessment  were  asked  to  complete  a questionnaire  about 
their  instructional  practices,  teaching  backgrounds,  and  characteristics.  The  teacher  questionnaire 
contained  two  parts.  The  first  part  pertained  to  the  teachers’  background  and  general  training.  The  second 
part  pertained  to  specific  training  in  teaching  mathematics  and  the  procedures  the  teacher  uses  for  each 
class  containing  an  assessed  student,  as  well  as  collecting  information  on  teachers’  awareness  and 
knowledge  of  the  NCTM  Standards. 

The  Teacher  Questionnaire,  Part  I:  Background  and  General  Training  included  questions 
pertaining  to  gender,  race/ethnicity,  years  of  teaching  experience,  certification,  degrees,  major  and  minor 
fields  of  study,  course  work  in  education,  course  work  in  specific  subject  areas,  amount  of  in-service 
training,  extent  of  control  over  instructional  issues,  and  availability  of  resources  for  their  classroom. 

The  Teacher  Questionnaire,  Part  II:  Training  in  Mathematics  and  Classroom  Instructional 
Information  included  questions  on  the  teacher’s  exposure  to  various  issues  related  to  mathematics  and 
teaching  mathematics  through  pre-  and  in-service  training,  ability  level  of  students  in  the  class,  whether 
students  were  assigned  to  the  class  by  ability  level,  time  on  task,  homework  assignments,  frequency  of 
instructional  activities  used  in  class,  methods  of  assessing  student  progress  in  mathematics,  instructional 
emphasis  given  to  the  mathematics  abilities  covered  in  the  assessment,  and  use  of  particular  resources. 

Because  the  sampling  for  the  teacher  questionnaires  was  based  on  participating  students,  the 
responses  to  a particular  teacher  questionnaire  do  not  necessarily  represent  all  teachers  of  that  subject 
area  at  that  grade  level  in  the  nation.  Rather,  they  are  teachers  of  the  representative  sample  of  students 
assessed.  It  is  important  to  note  that  in  all  NAEP  reports,  the  student  is  always  the  unit  of  analysis,  even 
when  information  from  the  teacher  or  school  questionnaire  is  being  reported.  Using  the  student  as  the 
unit  of  analysis  makes  it  possible  to  describe  the  instruction  received  by  representative  samples  of 
students.  Although  this  approach  may  provide  a different  perspective  from  other  studies  simply  reporting 
information  about  teachers  or  schools,  it  is  consistent  with  NAEP’s  goal  of  providing  information  about 
the  educational  context  and  performance  of  students. 

The  teacher  questionnaires  are  described  in  detail  in  Chapter  2. 


4.4.3  School  Questionnaires 

A School  Characteristics  and  Policies  Questionnaire  was  given  to  the  principal  or  other 
administrator  of  each  school  that  participated  in  the  assessment.  This  information  provided  an  even 
broader  picture  of  the  instructional  context  for  students’  mathematics  achievement.  This  questionnaire 
included  questions  about  background  and  characteristics  of  school  principals,  length  of  school  day  and 
year,  school  enrollment,  absenteeism,  dropout  rates,  size  and  composition  of  teaching  staff,  policies 
about  grouping  students,  curriculum,  testing  practices  and  uses,  special  priorities  and  school-wide 
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programs,  availability  of  resources,  special  services,  community  services,  policies  for  parental 
involvement,  and  school -wide  problems. 

School  Characteristics  and  Policies  questionnaires  are  described  in  detail  in  Chapter  2. 


4.4.4  SD/LEP  Student  Questionnaires 

The  SD/LEP  Student  Questionnaire  was  completed  by  the  teachers  of  those  students  who  were 
selected  to  participate  in  the  assessment  sample  who  were  classified  as  Students  with  Disabilities  (SD)  or 
were  classified  as  Limited  English  Proficient  (LEP).  The  questionnaire  was  completed  for  all  SD  or  LEP 
students,  whether  or  not  they  actually  participated  in  the  assessment.  This  questionnaire  asked  about  the 
nature  of  the  student’s  disability  and  the  special  programs  in  which  the  student  participated. 

Schools  were  permitted  to  exclude  certain  students  from  the  assessment.  The  same  exclusion 
criteria  and  rules  used  in  the  national  assessment  were  also  applied  to  the  Trial  State  Assessment. 
Although  the  intent  was  to  assess  all  sampled  students,  students  who  were  identified  by  school  staff  as 
not  capable  of  participating  meaningfully  were  excluded.  The  NAEP  guidelines  for  exclusion  were 
intended  to  assure  uniformity  of  exclusion  criteria  from  school  to  school  as  well  as  from  state  to  state. 

More  information  about  the  SD/LEP  questionnaire  and  exclusion  criteria  are  provided  in 
Chapters  2 and  5. 
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Chapter  5 


FIELD  OPERATIONS  AND  DATA  COLLECTION 

Lucy  M.  Gray,  Mark  M.  Waksberg,  and  Nancy  W.  Caldwell 

Westat,  Inc. 


5.1  INTRODUCTION 

This  chapter  describes  the  field  operations  and  data  collection  activities  for  the  national 
assessment  component  of  the  1996  National  Assessment  of  Educational  Progress  (NAEP).  The  national 
assessment  is  comprised  of  main  samples  and  long-term  trend  samples.  Main  NAEP  samples  typically 
involve  new  assessment  items,  and  may  include  new  subject  areas  and  innovative  features;  in  long-term 
trend,  the  procedures  and  items  from  previous  years  are  carried  forward  so  that  trends  in  student 
achievement  can  be  measured  over  time.  Both  the  main  and  long-term  trend  assessments  are  based  on 
probability  samples  of  schools  and  students  that  allow  for  regional  and  national  reporting  only.  The  State 
Assessment,  the  second  major  component  of  NAEP,  comprises  the  state  program  that  uses  main 
assessment  materials  and  involves  much  larger  sample  sizes  per  state  (or  jurisdiction),  so  that  results  can 
be  reported  for  each  participating  state  (for  further  technical  information  on  the  State  Assessment,  see  the 
Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Mathematics,  Allen,  Jenkins,  Kulick, 
and  Zelenak,  1997,  and  the  Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Science, 
Allen,  Swinton,  Isham,  and  Zelenak,  1997). 

The  design  of  the  national  assessment  component  of  NAEP  is  described  in  the  remaining  sections 
of  this  chapter.  For  all  components,  NAEP  guarantees  the  anonymity  of  participants,  and  student  or 
teacher  names  are  never  recorded  on  assessment  booklets  nor  removed  from  the  schools.  NAEP  results 
are  reported  on  the  national  level  and  by  region  of  the  country,  not  by  school  district,  school,  or 
individual  student.  Only  group  statistics  are  reported,  broken  down  by  gender,  race/ethnicity,  and  a host 
of  variables  that  illuminate  teachers’  instructional  practices. 


5.1.1  Field  Organization 

The  1996  main  assessment  involved  some  new  items  and  components  including  many  innovative 
features.  For  example,  the  science  assessment  differed  from  previous  NAEP  science  assessments  in  that 
every  student  performed  an  experiment.  The  mathematics  assessment  involved  the  use  of  mathematical 
tools  and  a larger  number  of  constructed-response  items  than  in  the  past.  Much  of  the  mathematics 
assessment  has  been  used  since  1990,  thus  providing  “short-term  trend”  data.  All  students  in  a particular 
assessment  session  received  a booklet  in  the  same  subject  (i.e.,  mathematics  or  science).  Even  though 
many  different  booklets  were  used  in  a particular  session,  they  were  all  for  the  same  subject. 

In  most  schools  sampled  for  the  main  assessment,  more  than  one  session  type  was  conducted.  In 
about  one-third  of  the  schools  at  each  grade,  only  two  session  types,  mathematics  and  mathematics 
estimation,  were  possible.  In  the  remaining  two-thirds  of  schools,  up  to  a maximum  of  four  session  types 


' Lucy  M.  Gray  and  Mark  M.  Waksberg  assist  in  survey  operations  and  field  activities  for  the  NAEP  assessments,  under  the 
direction  of  Nancy  W.  Caldwell. 
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for  grade  4 or  five  session  types  at  grades  8 or  12  could  have  been  conducted.  At  grade  12,  only  one 
advanced  session — either  mathematics  or  science — was  conducted. 

Historically,  a small  proportion  (less  than  10%)  of  the  sampled  students  have  been  “excluded” 
from  NAEP  assessment  sessions  because,  according  to  school  records,  they  are  students  with  either 
disabilities  or  limited  English  language  proficiency  who  have  been  determined  to  be  incapable  of 
participating  meaningfully  in  the  assessment.  More  recently,  especially  with  the  passage  of  the 
Individuals  with  Disabilities  Education  Act,  increased  attention  has  been  given  to  these  students  and  to 
including  as  many  of  them  as  possible  in  NAEP  sessions  (and  in  other  testing  situations  as  well).  NAEP 
has  addressed  these  concerns,  first  in  the  1995  field  test  and  continuing  with  the  1996  operational 
assessment,  through  a Special  Study  that  uses  both  old  and  new  “inclusion”  criteria  and  (in  some  schools) 
offers  accommodations  for  testing  students  with  disabilities  and/or  limited  English  proficiency 
(SD/LEP).  For  the  1996  main  assessment,  a split-sample  design  was  used,  placing  the  sampled  schools 
into  three  subsamples,  so  that  the  impact  of  both  the  new  SD/LEP  criteria  and  the  provision  of 
accommodations  could  be  evaluated,  while  also  collecting  data  with  the  old  criteria  to  maintain 
comparability  with  previous  NAEP  data  bases.  This  Special  Study  was  incorporated  in  the  main 
assessment  but  was  not  a part  of  the  1996  long-term  trend  assessment.  The  information  in  this  chapter 
and  in  Chapter  3 applies  to  all  three  sample  types  or  subsamples. 

For  administrative  purposes,  the  main  and  long-term  trend  assessments  were  conducted  in 
different  schools.  Responsibility  for  the  assessments  in  long-term  trend  schools  was  given  to  one  group 
of  assessment  supervisors,  while  responsibility  for  the  main  assessment  was  assigned  to  another  group  of 
supervisors.  Since  these  supervisors  worked  in  some  of  the  same  areas  and  sometimes  in  the  same  school 
districts,  careful  coordination  was  required. 

In  order  to  reduce  the  burden  on  the  participating  schools,  national  assessment  field  staff 
performed  most  of  the  work  associated  with  the  assessments.  Introductory  contacts  and  meetings  were 
held  in  the  fall  (1995)  to  enlist  cooperation  and  explain  the  assessment  procedures  to  district  and  school 
representatives  and  to  set  a mutually  agreed-upon  assessment  date  for  each  school.  The  assessment 
supervisor  visited  the  school  to  select  the  sample  of  students  a week  or  two  before  the  assessment.  The 
assessment  sessions  were  conducted  by  national  assessment  field  staff,  called  exercise  administrators, 
under  the  direction  of  the  assessment  supervisor.  At  the  conclusion  of  the  assessment  in  a school,  field 
staff  coded  demographic  information  on  the  booklet  covers  and  shipped  the  completed  materials  to 
National  Computer  Systems  (NCS),  the  processing  subcontractor  for  NAEP  (see  Chapter  6 for  more 
detailed  information  on  processing  assessment  materials). 


5.2  PREPARING  FOR  THE  ASSESSMENTS 
5.2.1  Gaining  the  Cooperation  of  Sampled  Schools 

The  process  of  gaining  cooperation  of  the  schools  selected  for  the  national  assessment  began  in 
late  August  1995  with  a series  of  letters  and  contacts  with  state  and  district-level  officials.  The  National 
Center  for  Education  Statistics  (NCES)  first  sent  each  jurisdiction  a letter  announcing  NAEP  plans  for 
1996.  Westat  then  contacted  the  State  Test  Directors  or  NAEP  State  Coordinators  in  each  sampled  state 
to  notify  them  of  the  districts  and  schools  selected  in  their  states.  In  the  40  jurisdictions  participating  in 
the  State  Assessment  that  also  had  schools  sampled  for  the  national  assessment,  the  state  received  the  list 
of  districts  and  schools  sampled  for  both  the  national  and  state  assessments. 
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From  September  through  early  December  1995,  Westat  sent  lists  of  schools  sampled  for  the  national 
assessment  component  and  other  NAEP  materials  to  district  superintendents,  diocesan  superintendents  of 
Catholic  schools,  and  principals  or  heads  of  schools  in  other  nonpublic  schools,  inviting  their 
participation.  These  initial  mailings  paved  the  way  for  telephone  contacts  by  NAEP  field  supervisors  who 
were  assigned  the  task  of  gaining  cooperation  and  scheduling  assessment  dates. 

The  schedule  for  project  activities  for  the  1996  main  and  long-term  trend  assessments  was  as 

follows: 


Date 

Mid"  August  1995 

August  27-31,  1995 

September  7-9,  1995 
Mid-September  1995 

Mid-to-Late  September  1995 

Mid-September  - December  1,  1995 


October  9 - December  22,  1995 
Early  December  1995 

December  9-13,  1995 
January  3 - March  29,  1996 
January  3 - March  8,  1996 
March  1 1 - May  10,  1996 


Activity 

Department  of  Education  sent  first  letter  to  Chief  State 
School  Officers  about  the  1996  assessment. 

Training  sessions  were  held  for  long-term  trend  assessment 
supervisors. 

Training  sessions  were  held  for  main  assessment  schedulers. 

Westat  sent  state  coordinators  a list  of  their  schools  initially 
selected  for  either  or  both  major  components. 

Westat  sent  samples  and  informational  materials  to  districts 
if  not  already  sent  by  state  coordinators. 

Supervisors  contacted  districts  and  schools  to  secure 
cooperation  and  to  schedule  assessments. 

Supervisors  conducted  introductory  meetings  for  the 
national  assessment,  by  telephone  (or  in  person  if  requested 
by  districts/schools).  Westat  selected  substitutes  for 
refusals. 

Supervisors  recruited,  hired,  and  trained  exercise 
administrators. 

Fall  long-term  trend  assessments  were  administered. 

Supervisors  sent  informational  materials  to  principals  and 
school  coordinators.  Letter  confirming  assessment  schedule 
sent  to  each  school  from  Westat. 

Main  assessment  supervisor  training  session  was  held. 

Main  assessments  were  administered.^ 

Winter  long-term  trend  assessments  were  administered. 

Spring  long-term  trend  assessments  were  administered. 


5.2.2  Supervisor  Training 

Training  for  assessment  supervisors  was  multi-phased  and  involved  separate  sessions  conducted 
in  August,  September,  and  December  1995.  All  training  was  conducted  by  the  Westat  project  director. 
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Final  makeup  sessions  were  held  April  1-5,  1996. 
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field  director,  and  home  office  staff.  Also  in  attendance  were  representatives  from  Educational  Testing  . 
Service  (ETS),  NCS,  and  NCES. 


The  first  of  these  training  sessions  was  held  August  27-31,  1995  in  Baltimore,  Maryland  for  field 
staff  assigned  to  the  long-term  trend  program  for  1996.  Attending  the  session  were  the  long-term  trend 
field  manager,  the  1 1 field  supervisors  responsible  for  conducting  the  long-term  trend  assessments,  and  4 
troubleshooters. 

After  an  introduction  to  the  study,  which  included  the  background  and  history  of  NAEP,  an 
overview  of  the  long-term  trend  assessments,  and  the  1995-1996  assessment  schedule,  the  training 
continued  with  a thorough  (2  half-day  sessions)  presentation  of  NAEP  contact/gaining  cooperation 
activities.  This  is  a lengthy  process  of  contacting  states,  districts,  and  schools  regarding  their 
participation  in  and  scheduling  for  NAEP;  several  demonstration  phone  calls,  role  plays,  and  exercises 
were  used  to  provide  some  practical  experience  during  this  part  of  the  training.  The  long-term  trend  staff 
was  also  trained  on  setting  assessment  schedules,  recruiting/hiring/training  exercise  administrators,  and 
sample  selection  and  preparation  of  Administration  Schedules  and  other  assessment  materials.  Several 
practice  exercises  were  used  to  demonstrate  these  topics.  The  training  concluded  with:  discussions  of 
conducting  the  session  and  using  the  session  script;  preparing  school  worksheets  and  holding  makeup 
sessions;  post-assessment  activities;  and  Westat  administrative  procedures. 

After  an  overview  of  NAEP  and  introductory  remarks  on  the  study  schedule,  the  main  assessment 
group  of  about  25  supervisors  received  extensive  training  (similar  to  the  August  training  for  long-term 
trend)  in  contacting  the  schools,  gaining  cooperation  and  scheduling  the  assessments;  numerous 
demonstrations,  role  plays  and  exercises  were  used.  Other  training  topics  included:  supervisory 
responsibilities;  setting  the  assessment  schedule;  recruiting  and  training  exercise  administrators;  and 
administrative  forms  and  procedures.  The  scheduling  supervisors  also  received  a full  day  of  training  on 
using  the  reporting  system  installed  on  the  laptop  computers  assigned  to  each  of  them  for  the  gaining 
cooperation/scheduling  phase. 

The  75  NAEP  supervisors  who  were  responsible  for  main  assessment  activities  were  trained 
during  a third  session,  held  December  9-13,  1995.  Training  focused  on  a review  of  the  preliminary 
activities  during  the  fall  including  results  of  initial  contacts  with  districts  and  schools,  scheduling  of 
assessments,  the  status  of  exercise  administrators’  recruitment,  and  a thorough  discussion  of  assessment 
activities:  sampling  procedures;  inclusion  of  SD/LEP  students;  teacher  surveys;  providing  testing 
accommodations;  conducting  science  sessions;  and  administrative  forms  and  procedures.  Westat’ s 
classroom  management  videotape  was  also  shown  at  this  training  session. 

The  main  and  state  assessment  field  managers  were  present  at  the  December  session  to  support 
training  activities  and  answer  questions  concerning  districts  and  schools  that  fell  into  the  samples  for 
more  than  one  component  of  the  assessment.  Each  supervisor  also  met  with  the  person  who  completed 
the  scheduling  in  their  area,  as  a first  step  in  preparing  for  the  new  supervisors’  contacts  with  each  school 
(and  district,  if  needed). 


5.2.3  Contacting  Districts  and  Nonpublic  Schools 

Once  the  supervisors  were  trained  in  August  and  September,  they  began  working  on  obtaining 
cooperation.  In  states  participating  in  the  State  Assessment,  the  assessment  supervisor  first  spoke  with 
the  State  field  manager  to  determine  what  contacts,  if  any,  had  already  been  made  with  districts  about  the 
national  assessment.  The  approach  the  supervisors  took  when  calling  superintendents  depended  on 
whether  the  district  had  been  notified  about  NAEP  by  the  State  Coordinator  and  whether  the  district  also 
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had  schools  selected  for  the  State  Assessment.  For  districts  that  had  been  contacted  by  the  State 
Coordinator,  the  supervisor  began  by  referring  to  that  contact. 

In  previous  NAEP  assessments,  the  supervisors  offered  and  usually  held  “introductory  meetings” 
with  representatives  from  the  superintendents’  offices  and  the  selected  schools,  typically  the 
superintendent  and  the  principals.  These  served  as  both  an  introduction  to  NAEP  and  a presentation  on 
what  would  be  asked  of  the  school.  The  meetings  were  also  used  to  establish  a schedule  for  the  sampling 
visits  and  the  assessments  in  the  schools. 

However,  over  the  years,  these  meetings  have  become  somewhat  redundant  since  many  districts 
have  fallen  into  the  NAEP  sample  more  than  one  time.  It  has  also  become  more  and  more  difficult  to 
schedule  these  meetings,  as  district  and  school  officials  find  it  harder  to  allot  time  away  from  their 
offices.  Thus,  beginning  with  the  fall  1995  preparation  for  the  1996  study,  the  material  was  almost 
always  presented  to  the  superintendents  and  principals  during  telephone  calls  rather  than  in  formal 
meetings.  Generally,  only  if  an  in-person  meeting  was  specifically  requested  by  the  district  or  school 
officials,  or  if  the  supervisor  felt  that  there  was  a better  chance  of  convincing  a district  to  participate  in 
person,  was  such  a meeting  held. 

As  the  supervisors  contacted  superintendents,  principals,  and  nonpublic  school  officials  to 
introduce  NAEP  and  determine  the  schools’  cooperation  status,  they  completed  two  forms  and  entered 
the  school  status  in  the  receipt  control  system  installed  on  their  laptop  computers.  The  Results  of  Contact 
Form  was  completed  to  document  the  discussion  the  supervisor  had  with  each  administrator  concerning 
the  district’s  willingness  to  participate  and  any  special  circumstances  regarding  the  schools’  cooperation 
or  assessments. 

The  supervisor  also  completed  portions  of  a School  Control  Form.  This  form  was  preprinted  with 
the  number  and  types  of  assessment  sessions  assigned  to  the  school,  so  that  this  information  could  then 
be  shared  with  the  district/school  official.  Information  gathered  during  the  phone  call,  including  the  name 
of  the  person  designated  to  be  the  school  coordinator,  the  number  of  students  in  the  designated  grade, 
tentative  dates  for  the  sampling  visit  and  assessment,  and  other  information  that  could  have  some  bearing 
on  the  assessment,  was  recorded  on  the  form.  This  information  was  used  to  update  records  in  the  home 
office.  In  December,  the  forms  were  provided  to  the  supervisors  who  would  be  conducting  the 
assessments. 

A small  number  of  in-person  introductory  meetings  were  held.  The  New  York  City  and  Los 
Angeles  City  school  districts  have  previously  used  these  meetings  to  present  information  about  the  NAEP 
assessments  to  the  officials  of  all  the  selected  schools  and  to  encourage  their  participation,  and  wished  to 
continue  that  practice  for  the  current  assessment.  A small  number  of  other  school  districts  also  requested 
such  a meeting,  involving  representatives  from  their  selected  schools  so  that  they  would  have  a full 
understanding  of  what  the  assessments  entailed. 

During  the  telephone  presentation  or  the  introductory  meeting,  the  supervisor  discussed 
arrangements  for  the  assessments  with  representatives  from  each  school.  Within  the  weeks  scheduled  for 
the  PSU,  the  supervisor  had  the  flexibility  to  set  each  school’s  assessment  date  in  coordination  with 
school  staff.  The  staff  sometimes  expressed  preferences  for  a particular  day  or  dates  or  had  particular 
times  when  the  assessment  could  not  be  scheduled.  Their  preferences  or  restrictions  depended  on  the 
events  that  had  already  been  scheduled  on  their  school  calendar.  Using  this  information  from  the  schools, 
the  supervisors  set  up  the  assessment  schedule  for  each  PSU. 

The  supervisor  usually  learned  during  the  introductory  contact  whether  a school  required  some 
form  of  parental  notification  or  permission.  Three  versions  of  standard  NAEP  letters  were  offered  for  the 
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school’s  use,  and  each  letter  could  be  produced  for  selected  students  only  or  for  all  eligible  students.  The 
first  version  informs  parents  about  the  assessment.  The  second  assumes  parental  consent  unless  parents 
send  the  form  back  stating  that  they  do  not  want  their  child  to  participate  in  the  assessment.  The  third 
version  requires  that  parents  sign  and  return  the  form  before  students  can  be  assessed.  All  versions  of  the 
letter  were  available  to  the  schools,  although  when  the  issue  of  parental  permission  came  up  in 
discussion,  supervisors  offered  the  least  restrictive  version  that  met  the  requirements  of  the  school  or 
district.  In  addition,  Spanish  language  versions  of  the  parent  information  letter  were  made  available  to  the 
schools.  Schools  could  also  send  out  their  own  letters  and  notices  if  they  preferred  not  to  use  those 
offered  through  NAEP.  Information  on  whether  the  school  required  parent  letters  and  the  type  of  letter 
used  was  recorded  on  the  School  Control  Form. 


5.2.4  Recruiting,  Hiring,  and  Training  Exercise  Administrators 

During  the  fall,  while  the  supervisors  were  contacting  their  schools  and  scheduling  assessments, 
their  other  major  responsibility  was  to  recruit  and  hire  exercise  administrators,  who  would  administer  the 
assessment  sessions.  Exercise  administrators  were  recruited  from  many  sources.  Each  supervisor  was 
given  a PSU-by-PSU  computerized  list  of  exercise  administrators  and  other  field  staff  who  had  worked 
previously  on  education  studies  for  Westat.  People  who  had  served  as  exercise  administrators  before, 
with  good  evaluations  from  their  previous  supervisors,  were  usually  the  first  considered  for  hiring. 
Subsequently,  during  contacts  with  the  schools,  the  supervisors  asked  the  school  principals  and  other 
staff  to  recommend  potential  exercise  administrators.  These  referrals  were  frequently  retired  teachers  or 
substitutes.  Finally,  where  necessary,  ads  were  placed  in  local  newspapers  and  the  employment  service 
was  notified. 

Supervisors  were  told  that,  in  general,  four  to  five  exercise  administrators  should  be  hired  for 
each  PSU,  although  a variety  of  factors  might  influence  the  actual  number.  The  number  of  schools  in  a 
PSU,  the  size  of  the  student  sample  in  each  school,  distances  to  be  traveled,  the  geography  of  the  area, 
and  weather  conditions  during  the  assessment  period  were  all  factors  taken  into  consideration  by 
supervisors  in  developing  their  plan  for  hiring  exercise  administrators. 

A few  supervisors,  whose  NAEP  assignments  contained  contiguous  PSUs,  hired  the  same 
exercise  administrators  to  work  in  all  their  PSUs.  Other  supervisors,  whose  assignments  comprised  PSUs 
that  were  not  geographically  connected,  tended  to  hire  teams  of  exercise  administrators  for  each  PSU. 
Supervisors  were  encouraged  to  hire  locally  and  to  hire  individuals  with  teaching  experience  and  the 
ability  to  handle  classroom  situations. 

The  scheduling  supervisors,  all  of  whom  were  experienced  NAEP  supervisors,  had  complete 
responsibility  for  recruiting,  hiring,  and  training  all  of  the  exercise  administrators,  including  ones  who 
would  report  to  different  assessment  supervisors.  The  training  was  standardized  so  that  all  supervisors 
used  a prepared  script  and  exercises  to  train  the  exercise  administrators. 

Each  exercise  administrator  received  an  exercise  administrator  manual,  which  covered  the  full 
range  of  their  job  responsibilities.  After  studying  the  manual,  they  attended  a half-day  training  session. 
During  the  training,  the  supervisor  reviewed  all  aspects  of  the  exercise  administrators’  job  including 
preparing  materials,  booklets,  and  Administration  Schedules  for  assessments;  the  actual  conduct  of  the 
session;  post-assessment  collection  of  materials;  coding  booklet  covers;  recordkeeping;  and 
administrative  matters.  In  January  1996,  each  exercise  administrator  attended  a shorter,  refresher  training 
session,  conducted  by  the  assessment  supervisor,  to  gain  further  experience  with  the  auxiliary  materials, 
such  as  mathematics  manipulatives  and  science  kits,  to  be  used  in  specific  assessment  sessions. 
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5.3 


SELECTING  THE  STUDENT  SAMPLE  FOR  MAIN  NAEP 


5.3.1  Grade-Eligible  Sample 

After  securing  cooperation  from  the  school,  the  first  scheduled  visit  to  the  school  was  made  to 
select  the  sample  of  students  to  take  part  in  the  main  assessments,  and  to  conclude  the  arrangements  for 
the  actual  testing.  This  visit  was  made  in  January  by  the  supervisor  responsible  for  the  assessments  in  the 
school.  Upon  arriving  at  the  school  (rarely,  sampling  was  done  at  the  district  office  instead  of  in  the 
school),  the  supervisor  first  reviewed  the  list  of  grade-eligible  students  and  confirmed  with  the  school 
coordinator  that  all  eligible  students  were  listed.  If  any  eligible  students  were  omitted,  sampling  could  not 
proceed  until  the  list  was  completed. 

Using  the  computer-generated  Session  Assignment  Form  (SAF)  for  the  main  assessment,  which 
was  specific  to  the  school,  the  supervisor  selected  the  sample  of  students  to  be  assessed.  The  SAF 
documented  the  types  of  sessions  to  be  administered,  the  anticipated  number  of  students  to  be  assessed, 
the  expected  number  of  students  eligible  for  the  assessment,  and  a series  of  line  numbers  designating  the 
students  to  be  sampled.  Those  eligible  students  on  the  school’s  master  list  whose  line  numbers  were 
shown  on  the  SAF  were  selected  for  the  assessment.  After  making  sure  that  all  eligible  students  had  been 
listed,  the  supervisor  numbered  the  students  on  the  master  list.  If  the  total  number  of  eligible  students  was 
within  the  minimum  and  maximum  limits  indicated  on  the  SAF,  the  supervisor  could  proceed  to  select 
the  sample.  If  the  number  was  outside  the  limits,  the  supervisor  called  Westat  for  additional  sampling 
instructions.  With  either  the  original  instructions  or  revised  line  numbers,  the  supervisor  proceeded  to 
select  the  sample  of  students.  The  SAFs  provided  step-by-step  instructions  for  sampling,  indicating  not 
just  the  line  number  of  each  student  to  be  selected,  but  the  type  of  assessment  session  for  which  each 
student  was  selected. 

Once  students  were  assigned  to  sessions,  the  supervisor  and  exercise  administrators  filled  out  an 
Administration  Schedule  for  each  session.  The  Administration  Schedule  is  the  primary  control  document 
for  the  assessment.  It  is  used  to  list  each  sampled  student  and  is  the  only  link  between  booklets  and 
students.  The  sample  was  designed  so  that  about  30  students  were  assigned  to  each  session.  The 
supervisor  discussed  the  final  schedule  of  the  sessions  with  the  school  coordinator  and  the  date,  time,  and 
location  of  each  session  were  filled  in  on  the  Administration  Schedules.  Because  student  names  were 
recorded  on  the  Administration  Schedules,  those  forms  remained  in  the  schools  after  the  sample  was 
drawn. 


The  supervisor  then  asked  the  school  coordinator  to  identify  any  students  in  the  sample  with  an 
Individualized  Education  Program  (lEP)  (for  reasons  other  than  being  gifted  and  talented)  and/or  who 
were  designated  as  LEP.  Any  student  with  either  (or  both)  of  these  designations  was  to  be  indicated  on 
the  Administration  Schedules.  The  school  was  asked  to  complete  an  SD/LEP  Student  Questionnaire  for 
each  student  with  this  designation.  This  was  to  be  completed  by  a teacher,  counselor  or  other  school 
official  who  knew  the  designated  student  well. 

The  school  coordinator  was  also  asked  to  determine  whether  any  of  these  students  should  be 
excluded  from  NAEP  based  on  the  criteria  for  assessing  SD/LEP  students  (discussed  in  detail  in  Section 

3 

5.3.2).  Preliminary  results  indicate  that  less  than  half  of  the  students  with  SD  and/or  LEP  designations 
were  excluded  from  the  assessment:  If  the  school  coordinator  could  not  identify  the  excluded  students 
while  the  supervisor  was  at  the  school,  the  instructions  were  left  with  the  Coordinator  along  with  blank 
copies  of  the  SD/LEP  Student  Questionnaire.  In  those  cases,  the  Coordinator  consulted  with  other  school 


^The  criteria  differs  for  the  main  and  long-term  trend  assessments. 
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officials  and  informed  the  supervisor  as  to  who  was  to  be  excluded  when  he/she  returned  for  the 
assessment. 


At  the  end  of  the  sampling  visit,  if  requested  by  the  school,  the  supervisor  and/or  exercise 
administrators  made  lists  of  the  sampled  students  for  the  teachers  and/or  completed  appointment  cards 
notifying  students  about  their  assessment  schedule.  Teacher  notification  letters  were  also  prepared  in 
some  schools,  which  explained  the  assessment  and  listed  the  students  who  had  been  selected. 

5.3.2  Sampling  for  Special  Studies 

Two  special  studies,  requiring  added  steps  in  the  sampling  process,  were  included  in  the  main 
assessment  for  1996.  (The  special  studies  were  not  a part  of  the  long-term  trend  component.)  One  of 
these  special  studies  involved  students  eligible  for  advanced  mathematics  or  science  sessions.  The  other 
involved  applying  two  versions  of  the  SD/LEP  “inclusion”  criteria  for  NAEP  assessments  and,  in  some 
schools,  offering  accommodations  for  testing  students  designated  as  SD/LEP. 

O Advanced  Sessions 

Samples  of  advanced  mathematics  or  advanced  science  students  were  designated  by  separate 
series  of  line  numbers  on  the  SAP  as  was  done  for  the  other  session  types  scheduled  for  a school. 
However,  before  these  students  could  be  listed  on  the  Administration  Schedule  for  the  advanced  session, 
it  was  necessary  to  check  each  selected  student’s  eligibility  using  lists  of  students  in  advanced  courses 
prepared  by  the  school.  The  definitions  of  advanced  students  were  as  follows: 

• Grade  8 Mathematics:  students  enrolled  in  Algebra  1 or  beyond  at  anytime  during 
the  1995-96  school  year; 

• Grade  12  Mathematics:  students  enrolled  in  Algebra  3,  Pre-Calculus,  Calculus  and 
Analytic  Geometry,  Calculus  or  AP  Calculus;  and 

• Grade  12  Science:  students  enrolled  in  AP  Biology,  Chemistry  2 (AP),  Physics  1, 

Physics  2 without  Calculus,  and  Physics  2 (AP). 

The  advanced  sessions  were  available  only  at  grade  8 in  mathematics  and  grade  12  in 
mathematics  and  science.  Of  the  students  designated  for  advanced  sessions  as  per  the  line  numbers  on  the 
SAP,  only  those  who  were  also  eligible  (according  to  the  definitions  above)  were  actually  listed  on  the 
Administration  Schedules  for  the  advanced  sessions. 

One  further  sampling  step  was  applied  in  grade  8 and  grade  12  schools  regardless  of  whether  the 
school  was  actually  scheduled  for  an  advanced  session.  After  the  samples  for  all  sessions  were  selected, 
the  supervisors  compared  the  school’s  lists  of  advanced  students  against  the  students  selected  for  each 
session  to  determine  those  students  who  were  eligible  for  an  advanced  session.  This  did  not  mean  that  all 
eligible  students  would  take  the  advanced  session  (only  those  selected  and  eligible  for  an  advanced 
session  were  actually  assessed  at  the  advanced  level),  but  the  eligibility  status  was  then  recorded  on  the 
Administration  Schedules  for  all  sessions  to  provide  a source  of  information  on  the  extent  to  which  the 
school  offered  advanced  mathematics  and  science  courses. 
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<>  SD/LEP  Sampling  and  Inclusion  Criteria 


Because  of  increased  interest  throughout  the  education  community  in  assessing  as  many  students 
as  possible,  NAEP  has  begun  to  evaluate  the  effects  of  using  revised  criteria  for  inclusion  of  SD/LEP 
students  and  providing  testing  accommodations  that  are  usually  offered  to  these  students  by  their  schools. 
For  the  1996  study,  the  school  sample  was  divided  into  three  subsamples  by  the  statisticians  at  Westat. 
The  purpose  of  the  subsamples  was  to:  collect  data  under  the  same  conditions  as  previous  studies  in  order 
to  maintain  trend  in  mathematics  within  NAEP;  evaluate  the  impact  of  a revised,  more  specific  set  of 
inclusion  criteria;  and  evaluate  the  combined  effect  of  the  new  criteria  and  the  use  of  accommodations 
for  testing  students.  The  three  subsamples  of  schools  were  defined  as  follows: . 

51  (Sample  1)  These  schools  used  the  criteria  from  1990  and  1992,  and 

accommodations  were  not  provided.  Since  the  issue  of  “trend” 
applies  to  mathematics  and  not  to  science,  only  mathematics 
was  assessed  in  these  schools. 

52  (Sample  2)  These  schools  used  the  new  1996  criteria,  but  accommodations 

were  not  offered.  This  was  designed  to  evaluate  only  the  impact 
of  changing  the  criteria. 

53  (Sample  3)  These  schools  applied  the  new  1996.  criteria  and  the 

accommodations  most  commonly  used  for  achievement  testing 
were  offered.  For  LEP  students,  the  adaptations  included  a 
bilingual  (English-Spanish)  version  of  the  mathematics 
assessment  and  a Spanish-language  glossary  for  the  science 
assessment.  For  lEP  students,  the  accommodations  included: 
small  group  or  one-on-one  assessments,  untimed  assessments, 
and  reading  aloud.  Large-print  and  Braille  booklets  were  also 
offered  in  some  mathematics  sessions. 

During  the  sampling  visit,  after  the  samples  of  students  were  selected  for  each  session,  the 
schools  were  asked  to  identify  any  sampled  students  whom  the  school  considered.to  be  SD/LEP.  The 
school  was  then  asked  to  complete  an  SD/LEP  Questionnaire  regarding  each  of  these  students.  These 
basic  steps  were  consistent  with  previous  studies  and  did  not  vary  among  the  three  types  of  schools. 

Further,  the  schools  were  asked  to  indicate  which  of  the  SD/LEP  students  should  be  included  in 
the  assessment  and  which  should  not.  Again,  this  step  is  consistent  with  previous  studies,  but  in  1996,  the 
specific  criteria  and  availability  of  accommodations  varied  among  the  three  types  of  schools.  Each  school 
based  its  decisions  about  assessing  SD/LEP  students  on  the  specific  criteria  provided  to  that  school. 

To  produce  as  large  a sample  as  possible  of  students  from  which  to  evaluate  the  new  criteria  and 
procedures,  SD/LEP  students  were  oversampled  in  certain  session  types  in  some  schools.  This  was  the 
case  for  mathematics  sessions  at  grades  4,  8,  and  12  in  S2  schools  and  in  both  mathematics  and  science 
sessions  at  all  three  grades  in  S3  schools.  The  oversampled  students  were  added  to  the  appropriate 
Administration  Schedules  (according  to  the  instructions  on  the  SAF)  as  part  of  the  sampling  process. 

The  SAF  contained  specific  instructions  on  oversampling  for  the  NAEP  supervisor  if  it  was 
required  in  a particular  school.  Oversampling  of  SD/LEP  students  was  performed  only  in  S2  and  S3 
schools,  and  the  “pool”  from  which  the  oversample  was  selected  varied  according  to  the  grade  to  be 
assessed  and  the  types  of  sessions  scheduled. 


130 


105 


The  pool  was  defined  as: 


• Grade  4 - any  lEP  and/or  LEP  students  from  the  grade-eligible  list  who  were  not 
selected  for  any  session  type;  or 

• Grade  8 or  12  with  an  advanced  session  - any  lEP  and/or  LEP  students  preselected 
for  an  advanced  session  but  not  eligible  for  that  session;  or 

• Grade  8 or  12  with  no  advanced  session  - any  EBP  and/or  LEP  students  from  the 
grade-eligible  list  who  were  not  selected  for  any  session  type. 

For  the  supervisors  to  select  the  oversample  correctly,  it  was  necessary  to  first  complete  the  sampling  for 
every  session  in  the  school  and  to  be  sure  that  the  school  had  provided  EBP  and/or  LEP  status  for  every 
student  on  the  grade-eligible  list  (i.e.,  not  just  for  the  selected  students). 

Once  the  oversample  pool  was  established  for  a particular  school,  the  supervisor  numbered 
(consecutively)  the  students  in  the  pool.  (This  was  essentially  a renumbering  and  was  done  separately 
from  the  original  numbering  of  all  students  on  the  list).  The  oversample  was  then  selected  using  the 
oversampling  line  numbers  specified  on  the  SAF.  Students  were  added  to  either  regular  mathematics  or 
regular  science  sessions  according  to  the  specifications  on  the  SAF.  The  names  of  the  oversampled 
students  were  inserted  at  the  end  of  the  appropriate  Administration  Schedule.  The  school  was  asked  to 
complete  an  SD/LEP  Questionnaire  on  each  such  student  and  to  determine  whether  the  student  should  be 
included  in  the  assessment  and,  for  S3  schools,  what  specific  testing  accommodation(s),  if  any,  are  called 
for  in  the  students  lEP  and/or  are  normally  provided  for  each  student  by  the  school. 

The  unweighted  results  of  the  1996  assessments  show  that  the  sampling  process  generated,  in 
total,  15,871  students  to  be  assessed  in  SI  schools,  48,769  to  be  assessed  to  S2  schools,  and  41,513  in  S3 
schools.  These  counts  include  the  SD/LEP  students  that  the  schools  determined  should  participate  in  the 
assessments.  Accommodations  were  used  in  just  over  200  S3  schools  (about  30%)  for  approximately 
1,050  students.  The  most  frequently  provided  accommodations  were  small  group,  extended  time 
(untimed  testing),  and  bilingual  assessment  booklets.  These  results  are  very  preliminary,  however, 
because  they  are  unweighted  and  cannot  be  used  to  compare  results  for  SI,  S2,  and  S3  schools  (without 
applying  the  weighting  process).  Detailed  information  and  results  of  the  SD/LEP  special  study  will  be 
provided  in  a separate  report. 


5.4  CONDUCTING  THE  ASSESSMENT  SESSIONS 

The  primary  responsibility  for  conducting  assessment  sessions  was  given  to  the  exercise 
administrators.  Supervisors  were  required  to  observe  the  first  session  each  exercise  administrator 
conducted  to  ensure  that  they  followed  the  procedures  properly.  Supervisors  were  also  required  to  be 
present  in  all  schools  with  more  than  one  small  session  to  be  conducted.  The  supervisor  plays  an 
important  role  as  the  liaison  between  the  national  assessment  and  school  staff  ensuring  that  the 
assessments  go  smoothly. 

To  ensure  that  sessions  were  administered  in  a uniform  way,  the  exercise  administrator  was 
provided  with  scripts  for  each  session  type.  The  scripts  were  to  be  read  verbatim.  The  scripts  began  with 
a brief  introduction  to  the  study.  The  exercise  administrator  was  then  directed  to  distribute  the  booklets, 
being  careful  to  match  the  student  with  the  preassigned  booklet. 
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After  the  booklets  were  distributed,  some  additional,  scripted  directions  were  read.  Students  were 
asked  to  write  in  the  NAEP  school  ID  (except  in  grade  4,  where  NAEP  staff  entered  the  ID)  and  their 
home  ZIP  code  on  the  cover  of  the  booklet,  and  given  some  general  directions  in  completing  the 
assessment.  For  fourth  grade  students,  all  of  the  background  questions  were  then  read  aloud  by  the 
exercise  administrator;  at  the  upper  grades,  the  first  question,  which  asks  the  students’  race/ethnicity,  was 
read  by  the  exercise  administrator,  and  the  students  read  the  rest  to  themselves.  After  the  background 
questions  were  completed,  the  students  were  told  that  any  further  questions  they  might  have  could  not  be 
answered  by  the  exercise  administrator,  and  that  they  were  to  begin  the  first  cognitive  section  of  the 
assessment.  This  process  (along  with  the  script)  was  modified  somewhat  for  science  where  the 
background  questions  were  at  the  end  of  the  assessment  booklet,  and  none  of  the  items  was  read  aloud  at 
grades  8 or  12. 

During  the  sessions,  the  exercise  administrators  walked  around  the  room  monitoring  the  students 
to  make  sure  they  were  working  in  the  correct  section  of  their  booklet  and  to  discourage  them  from 
looking  at  a neighbor’s  booklet. 

At  the  end  of  each  assessment  session,  booklets  were  collected  and  students  dismissed  according 
to  the  school’s  policy.  The  exercise  administrator  was  then  responsible  for  completing  the  information  at 
the  top  of  the  Administration  Schedule,  totaling  the  number  of  participating  students,  and  coding  the 
covers  of  all  booklets,  including  those  booklets  assigned  to  absent  students. 


5.5  RESULTS  OF  THE  MAIN  NAEP  ASSESSMENT 
5.5.1  School  and  Student  Participation 

The  unweighted  school  response  rate  for  the  main  assessments  in  1996  was  82  percent.  The  final 
sample  of  cooperating  schools  included  604  schools  at  grade  4,  592  schools  at  grade  8,  and  591  schools 
at  grade  12.  Table  5-1  shows  comparative  response  rates  for  the  last  four  assessment  periods. 

Unlike  the  student  response  rates,  there  has  been  a small  but  steady  decline  in  the  main 
assessment  school  response  rates  over  the  last  several  assessment  periods.  This  has  occurred  despite 
persistent  efforts  to  convert  schools  and  districts  that  indicate  that  they  are  not  interested  in  participating 
in  the  assessments.  Both  Westat  field  managers  and  ETS  staff  have  been  employed  in  these  conversion 
efforts. 


Table  5-1 

Comparison  of  Student  and  School  Response  Rates  for  Main  NAEP,  1990-1996 


1990 

1992 

1994 

1996 

Student  response 

Grade  4 

92.9 

93.4 

93.2 

95.5 

Grade  8 

89.0 

88.8 

91.0 

93.2 

Grade  12 

80.8 

80.8 

81.1 

80.1 

School  response 

Grade  4 

88.3 

86.4 

86.0 

85.8 

Grade  8 

86.7 

85.3 

85.5 

81.9 

Grade  12 

81.3 

81.5 

78.6 

78.7 
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The  most  frequently  stated  reason  for  school  and  district  refusals  has  been  the  increase  in  testing 
throughout  the  jurisdictions  and  the  resulting  difficulty  in  finding  time  in  the  school  schedule  to  conduct 
the  NAEP  assessments.  With  so  many  states  now  mandating  their  own  testing,  school  schedules  are 
becoming  tighter,  and  administrators  are  finding  it  increasingly  difficult  to  accommodate  outside  testing. 
Despite  the  increased  visibility  and  publicity  surrounding  NAEP,  schools  are  reluctantly  finding  it 
necessary  to  decline  participation  as  a result  of  the  increasing  demands  on  their  students’  time. 

Of  the  1 13,846  students  sampled  for  the  1996  assessment,  roughly  five  percent  overall  were 
excluded  by  schools.  Altogether,  94,157  students  were  assessed  across  all  three  grades:  28,528  students 
were  assessed  at  fourth  grade,  32,339  were  assessed  at  eighth  grade,  and  33,290  students  were  assessed  at 
twelfth  grade.  The  overall  student  participation  rate  was  88.7  percent  (after  eliminating  any  withdrawn 
and  excluded  students). 

The  response  rate  at  which  supervisors  were  required  to  conduct  a makeup  session  was  raised 
from  the  standard  that  had  been  used  in  previous  main  assessments.  The  previous  rates  of 
75  percent  and  85  percent  (in  1994  only)  were  changed  to  90  percent  for  1996,  that  is,  any  session  (or 
group  of  sessions  within  the  same  subject  area)  at  which  fewer  than  90  percent  of  the  eligible  students 
were  assessed  would  require  a makeup  session  (assuming  that  the  school  was  willing  to  schedule  one). 
This  change  resulted  in  129  schools  conducting  makeup  sessions  that  would  not  have  been  required  to  do 
so  under  the  85  percent  rule.  In  these  schools,  an  additional  595  students  were  assessed.  These  595 
students  served  to  increase  the  overall  response  by  less  than  one  percentage  point  (0.6%).  The  greatest 
increase  was  at  the  grade  12  level,  where  413  students  were  assessed  in  these  additional  sessions,  which 
raised  the  response  rate  for  this  group  of  students  by  about  one  percentage  point. 


5.5.2  Assessment  Questionnaires 

Westat  provided  each  school  with  a School  Characteristics  and  Policies  Questionnaire  a few 
weeks  before  the  assessment  was  scheduled  to  be  conducted  (i.e.,  at  the  time  of  sampling).  At  the  same 
time,  supervisors  prepared  an  SD/LEP  Student  Questionnaire  for  each  sampled  student  with  either  an  lEP 
and/or  LEP  designation,  with  the  request  that  it  be  completed  by  someone  at  the  school  knowledgeable 
about  that  student. 

Selected  teachers  of  fourth-  and  eighth-grade  mathematics  and  science  were  asked  to  fill  out 
Teacher  Questionnaires.  The  teachers  asked  to  participate  were  the  mathematics  or  science  teachers  of 
those  students  selected  for  the  assessment  so  that  the  teacher  data  could  be  linked  to  student  performance 
data.  The  Teacher  Questionnaire  for  grade  4 was  combined  into  one  form  since  it  is  recognized  that  at 
this  grade  level,  the  same  teacher  would  probably  teach  all  of  the  subjects.  For  grade  8,  there  were  two 
distinct  questionnaires,  one  for  mathematics  teachers  and  the  other  for  science  teachers.  At  grade  12,  a 
teacher  questionnaire  was  used  only  for  advanced  mathematics  sessions. 

The  supervisor  requested  that  the  Teacher  Questionnaires  be  distributed  as  quickly  as  possible 
after  the  sampling  so  that  they  could  be  returned  by  the  day  of  the  assessment.  Additional  introductory 
materials  were  included  with  the  Teacher  Questionnaires  in  response  to  questions  that  teachers  have  had 
in  the  past  about  the  importance  of  completing  the  questionnaires  and  about  NAEP  in  general.  Teachers 
received  a letter  explaining  the  purpose  of  the  Questionnaire,  along  with  background  materials  about 
NAEP. 


If  the  Teacher  Questionnaires  were  not  complete  at  the  time  of  the  assessment,  the  supervisor  left 
a postage-paid  envelope  to  NCS  to  be  used  to  return  the  questionnaires.  Table  5-2  shows  the  number  of 
questionnaires  distributed  and  the  number  completed. 
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Table  5-2 

Background  Questionnaires  Received  for  Schools^  Teachers, 
and  SD/LEP  Students  in  the  1996  Main  Assessment^ 


...  Teacher  Questionnaires 

Characteristics 


Grade 

Assessed 

and  Policy 
Questionnaire 

Mathematics/Science 
(grade  4 only) 

Mathematics 

Science 

SD/LEP  Student 
Questionnaire 

Grade  4 

Number  expected 

605 

1,601 

NA 

NA 

5,116 

Number  received 

577 

1,601 

NA 

NA 

4,885 

Percent  received 

95% 

100% 

NA 

NA 

95% 

Grade  8 

Number  expected 

592 

NA 

1,400 

844 

5,048 

Number  received 

554 

NA 

1,365 

802 

4,770 

Percent  received 

94% 

NA 

98% 

95% 

94% 

Grade  12^ 

Number  expected 

593 

NA 

475 

NA 

4,147 

Number  received 

546 

NA 

475 

NA 

3,806 

Percent  received 

92% 

NA 

100% 

NA 

92% 

' The  numbers  in  this  table  reflect  the  full  samples,  including  SI , S2,  and  S3. 

^ At  grade  12,  teacher  questionnaires  were  used  only  for  teachers  of  advanced  mathematics.  Thus,  no  data  were  collected  from  science  teachers, 
and  the  data  shown  here  represent  teachers  of  advanced  mathematics  only. 


5.6  LONG-TERM  TREND  ASSESSMENTS 
5.6.1  Overview 

To  provide  continuity  and  comparability  with  past  NAEP  studies,  the  long-term  trend  component 
(formerly  referred  to  as  the  “bridge”  assessments)  replicates  procedures  and  materials  that  have  been 
used  since  the  inception  of  NAEP.  Student  eligibility  in  long-term  trend  is  always  based  on  criteria  used 
in  years  prior  to  1988  (when  the  modal  grade  for  students  aged  17  changed  from  the  grade  1 1 to  grade 
12).  The  1996  schedule  for  long-term  trend  assessments  was  as  follows:  the  fall  assessment  of  age 
13/grade  8 students  was  held  in  the  11-week  period  from  October  9 through  December  22,  1995;  the 
winter  assessment  of  age  9/grade  4 students  was  held  during  the  10-week  period  from  January  3 through 
March  8,  1996;  and  the  spring  trend  assessment  of  students  who  were  age.  17/grade  11  was  conducted  in 
the  9-week  period  from  March  11  through  May  10,  1996.  Students  were  assessed  in  reading,  writing, 
mathematics,  and  science. 

Paced  tape  sessions  were  conducted  with  samples  of  age-eligible  students  only,  as  was  done  in 
all  previous  years.  Additional  samples  of  age-  and  grade-eligible  students  were  assessed  with  spiral 
(print-administered)  booklets,  following  procedures  initiated  in  1984.  Six  different  types  of  sessions  were 
conducted:  one  print-administered  and  five  separate  tape-administered  sessions.  Depending  on  the  size  of 
the  school,  up  to  four  different  session  types,  involving  a total  of  about  80  students,  might  have  been 
conducted  in  a participating  school. 
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5.6.2  Selecting  the  Student  Sample 


Procedures  for  sampling  in  long-term  trend  schools  were  very  similar  to  those  employed  in  the 
schools  selected  for  the  main  assessment.  One  to  two  weeks  before  the  assessment,  the  supervisor  visited 
the  school  to  select  the  sample.  Lists  of  students  were  reviewed  to  ensure  that  all  age-  and  grade-eligible 
students  were  listed.  The  SAP  for  long-term  trend  schools  specified  a range  for  the  expected  number  of 
eligible  students.  If  the  total  number  of  students  was  within  this  allowable  range,  the  sampling  could 
proceed.  Otherwise,  the  supervisor  called  Westat  for  additional  sampling  instructions.  The  SAP  directed 
the  supervisor  to  assign  students  to  long-term  trend  session  types  based  on  their  line  numbers  from  the 
student  list  that  the  school  had  prepared.  (The  SAPs  for  the  long-term  trend  sample  were  like  those  used 
in  long-term  trend  sampling  for  previous  years,  and  were  distinct  from  the  SAPs  for  the  main  assessment 
that  were  described  earlier  in  this  report.) 

The  only  major  variation  within  the  sampling  for  long-term  trend  assessments  was  that,  for  the 
tape  sessions,  only  age-eligible  students  were  selected.  Por  these  sessions,  the  supervisor  selected  from 
the  entire  list  of  students  (age-  and  grade-eligible),  but  then  deleted  those  who  were  only  grade-eligible 
before  recording  the  names  of  the  students  to  be  assessed  on  the  Administration  Schedules. 

The  criteria  for  excluding  students  were  also  different  for  the  long-term  trend  schools  (compared 
to  the  main  assessment),  and  again  followed  the  criteria  that  were  established  previously  for  long-term 
trend.  Por  those  students  who  were  excluded,  the  school  was  asked  to  complete  an  Excluded  Student 
Questionnaire.  If  the  school  coordinator  could  not  identify  the  excluded  students  while  the  supervisor 
was  at  the  school,  a set  of  instructions  for  excluding  students  was  left  with  the  coordinator  along  with  the 
estimated  number  of  questionnaires  that  would  be  needed. 


5.6.3  Conduct  of  the  Assessment 

The  conduct  of  the  assessments  in  schools  selected  for  the  long-term  trend  program  is  essentially 
the  same  as  in  the  schools  selected  for  the  main  assessment.  Scripts  are  provided  for  the  supervisors  and 
exercise  administrators  to  use  in  administering  the  sessions.  The  major  difference  compared  to  main 
assessment  is  that  most  of  the  sessions  are  tape  administered.  In  these  sessions,  after  the  distribution  of 
the  test  booklets,  the  administrator  is  instructed  to  turn  on  the  tape  recorder.  The  remainder  of  the 
instructions  are  contained  on  the  tape,  and  the  timing  is  determined  by  the  length  of  time  that  the  tape 
runs. 


5.6.4  Results  of  Long-Term  Trend  Assessments 

The  unweighted  school  response  rate  for  the  1996  long-term  trend  assessments  was  83  percent. 
The  final  sample  of  cooperating  schools  included  240  schools  at  age  9/grade  4,  and  238  schools  at  age 
13/grade  8,  and  191  schools  at  age  17/grade  1 1.  Nearly  30,000  students  were  assessed  in  long-term  trend, 
or  91  percent  of  those  eligible  to  be  assessed. 

Of  the  36,371  students  sampled  for  long-term  trend  assessments,  eight  percent  were  excluded  by 
the  schools.  Overall,  29,791  students  were  assessed  across  all  three  age/grade  groups:  10,432  students 
were  assessed  at  age  9/grade  4,  1 1,151  students  were  assessed  at  age  13/grade  8,  and  8,208  were  assessed 
at  age  17/grade  11.  Table  5-3  shows  comparative  response  rates  for  the  last  four  long-term  trend 
assessments. 
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Table  5-3 

Comparison  of  Student  and  School  Response  Rates  for  Long-Term  Trend  NAEP,  1990-1996 


1990 

1992 

1994 

1996’ 

Student  response 

Grade  4 

92.4 

94.0 

94.2 

95.5 

Grade  8 

90.4 

90.8 

92.2 

91.9 

Grade  12 

81.2 

82.8 

84.1 

84.0 

School  response 

Grade  4 

88.1 

87.4 

86.7 

84.8 

Grade  8 

90.5 

84.7 

81.7 

82.4 

Grade  12 

80.7 

81.3 

81.1 

81.3 

' The  numbers  in  this  table  reflect  the  full  samples,  including  S 1 , S2,  and  S3. 


5.6.5  Assessment  Questionnaires 

The  School  Characteristics  and  Policies  Questionnaire  and  the  Excluded  Student  Questionnaire 
are  forms  that  were  distributed  in  the  schools  to  be  completed  by  school  personnel.  The  School 
Characteristics  and  Policies  Questionnaire  was  provided  to  the  school  by  the  assessment  supervisor  at  the 
time  of  the  sampling  visit.  This  form  was  to  be  filled  out  by  the  principal  or  other  staff  member 
knowledgeable  about  the  school’s  administrative  policies  and  staff  characteristics.  The  supervisors 
collected  the  completed  questionnaire  when  they  returned  to  the  school  for  the  assessment. 

An  Excluded  Student  Questionnaire  was  to  be  filled  out  for  every  student  who  was  sampled  for 
the  assessment  but  excluded  by  the  school.  Following  exclusion  criteria  used  in  previous  long-term  trend 
assessments,  schools  could  exclude  students  with  limited  English-speaking  ability,  those  who  were 
educable  mentally  retarded,  or  functionally  disabled  students,  if  in  the  judgment  of  school  staff  or  as 
indicated  in  school  records,  they  were  unable  to  “participate  meaningfully”  in  the  assessment.  After  the 
sample  of  students  was  drawn  and  Administration  Schedules  prepared,  the  supervisor  requested  that  the 
school  coordinator  identify  any  students  who  should  be  excluded.  The  supervisor  then  gave  an  Excluded 
Student  Questionnaire  to  the  coordinator  for  every  excluded  student,  with  the  request  that  it  be  completed 
by  someone  in  the  school  knowledgeable  about  the  student.  (Note  that  this  varies  somewhat  from  the 
main  assessment  where  questionnaires  are  assigned  for  all  sampled  students  with  an  lEP  and/or  LEP,  not 
Just  for  those  who  are  excluded  from  the  assessment.) 

The  supervisor  attempted  to  collect  all  completed  questionnaires  (School  Characteristics  and 
Policies  Questionnaire  and  Excluded  Student)  on  the  assessment  day.  If  the  questionnaires  were  not 
ready,  and  it  was  convenient  for  the  supervisor  or  an  exercise  administrator  to  return  to  the  school  later  to 
pick  them  up,  they  would  do  so.  Otherwise,  the  supervisor  gave  the  coordinator  a postage-paid  envelope 
to  use  to  mail  the  forms  to  NCS.  All  (100%)  of  the  School  Characteristics  and  Policies  Questionnaires 
were  completed  and  returned,  and  95.9  percent  of  the  Excluded  Student  Questionnaires  were  returned. 

Once  the  assessments  were  completed  in  a school,  the  supervisor  and  exercise  administrators 
completed  the  coding  of  the  front  covers  of  the  assessment  booklets,  filled  out  the  necessary  forms,  and 
shipped  the  booklets  and  forms  to  NCS.  A copy  of  all  forms  was  sent  to  Westat  so  that  progress  in  the 
field  could  be  closely  monitored. 

The  School  Worksheet  was  used  by  the  supervisor  to  summarize  the  results  of  the  assessment 
sessions  in  each  school.  The  number  of  students  to  be  assessed,  the  number  actually  assessed,  and  the 
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number  absent  were  entered  so  that  the  supervisor  could  calculate  whether  a makeup  session  was 
required.  Attendance  of  less  than  90  percent  required  a makeup.  If  a makeup  was  required  for  one  or 
more  session  types,  the  supervisor  discussed  the  scheduling  of  the  makeup  with  the  school  coordinator. 

In  long-term  trend  assessments  prior  to  1994,  the  percentage  of  students  attending  that  would 
necessitate  a makeup  session  was  75  percent  or  below.  For  1994,  this  rate  was  increased  to  85  percent, 
and  it  was  raised  again  to  90  percent  for  1996.  By  raising  the  rate  to  90  percent,  an  additional  144 
students  were  assessed  in  73  schools,  compared  to  the  85-percent-rule,  increasing  the  overall  student 
response  rate  for  1996  by  less  than  one-half  of  one  percentage  point  (0.44%). 

The  top  (original)  copy  of  the  School  Worksheet,  any  Excluded  Student  Questionnaires 
completed  by  the  school,  and  the  Administration  Schedules  (with  the  students’  names  removed  and  left  at 
the  school)  were  included  with  the  booklets  in  the  shipment  to  NCS.  In  addition,  the  supervisor  included 
a packing  list  with  the  materials,  which  inventoried  the  assessment  materials  assigned  to  and  returned 
from  the  school. 


5.7  FIELD  MANAGEMENT 

Two  field  managers  monitored  the  work  of  25  scheduling  supervisors  who  worked  during  fall  ' 
1995  to  gain  cooperation  of  districts  and  schools  for  the  main  assessment.  During  the  assessment  period, 
these  staff  were  expanded  to  about  75  supervisors  and  6 field  managers  (4  of  whom  were  located  in 
Westat’s  home  office).  An  additional  field  manager  was  assigned  exclusively  to  long-term  trend  NAF.P 
throughout  the  gaining  cooperation  and  assessment  periods.  All  supervisors  reported  directly  to  their 
field  managers  who,  in  turn,  reported  to  Westat’s  field  director.  All  contacts  were’ made  at  least  weekly. 

An  automated  management  system  was  developed  and  maintained  in  Westat’s  home  office.  The 
scheduling  supervisors  working  to  contact  schools  during  the  fall  used  this  system  on  their  portable 
computers.  The  system  contained  a record  for  each  sampled  school.  A disposition  code  structure  was 
developed  to  indicate  the  status  of  each  school’s  participation  (e.g.,  school  cooperating,  decision  pending, 
school  refusal,  district  refusal,  school  closed,  etc.).  As  a school’s  status  was  determined,  the  scheduling 
supervisors  entered  the  status  of  the  school  onto  their  computers,  and  this  information  was  downloaded 
onto  the  home  office  system  on  a weekly  basis.  Disposition  reports  were  then  generated  from  the  receipt 
system  once  a week  so  that  home  office  staff  could  review  the  progress  of  securing  cooperation  from  the 
sampled  schools. 

These  reports  were  an  invaluable  tool  for  the  sampling  statisticians  as  well  as  for  the  field 
director  and  field  management  staff.  They  provided  the  statisticians  with  the  information  needed  to 
determine  whether  the  sample  of  schools  was  adequate  to  produce  representative  results.  Based  on  the 
information  contained  in  these  reports,  the  sampling  statisticians  selected  substitute  schools  to  replace 
some  of  the  non-cooperating  schools. 

After  assessments  were  completed,  the  system  was  used  to  enter  data  from  the  School 
Worksheets  (for  both  Main  NAEP  and  long-term  trend)  on  the  number  of  students  to  be  assessed,  the 
number  assessed,  and  the  nurriber  absent  for  each  school.  Data  on  completed  questionnaires  received  was 
provided  by  NCS.  The  system  was  also  used  to  alter  school  assessment  dates,  particularly  when  bad 
weather  required  a change  in  schedule,  and  to  monitor  plans  for  and  progress  in  conducting  makeup 
sessions.  Reports  were  generated  weekly  during  the  assessment  period  that  allowed  the  project  staff  to 
monitor  the  progress  of  the  assessments  both  in  terms  of  checking  that  the  schools  were  assessed  on 
schedule  as  well  as  assuring  that  a high  response  rate  was  achieved.  The  sampling  statisticians  used  these 
reports  to  monitor  the  sample  yield  by  school,  PSU,  and  age/grade  level. 
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Progress  of  the  assessments  was  constantly  monitored  through  telephone  reports  held  between 
NAEP  supervisors,  field  managers,  and  home  office  staff.  During  these  phone  conversations,  the 
supervisors’  schedules  were  reviewed  and  updated,  and  any  problems  that  the  supervisors  were 
experiencing  were  discussed.  Much  of  the  attention  this  year  was  focused  on  maintaining  the  schedule  in 
light  of  the  many  postponements  due  to  severe  winter  weather. 

The  supervisors  who  traveled  filled  out  a Work  Schedule  for  a one-  to  two-week  period,  showing 
their  whereabouts,  so  that  they  could  be  contacted  if  necessary.  It  also  allowed  field  managers  and  project 
staff  to  review  the  supervisors’  schedules  and  the  distribution  of  work. 

Progress  of  the  field  work  was  also  monitored  during  quality  control  visits  made  to  the  field  by 
Westat  and  ETS  office  staff. 
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Chapter  6 

PROCESSING  ASSESSMENT  MATERIALS^ 


Patrick  B.  Bourgeacq,  Bradley  Thayer,  and  Timothy  Robinson 
National  Computer  Systems 


6.1  INTRODUCTION 

This  portion  of  the  report  reviews  the  activities  conducted  by  National  Computer  Systems  (NCS) 
for  the  NAEP  1996  main  assessments  in  mathematics  and  science,  and  long-term  trend  assessments  in 
reading,  mathematics,  science,  and  writing.  As  a subcontractor  to  Educational  Testing  Service  (ETS), 

NCS  was  responsible  for  printing  all  of  the  NAEP  student  booklets  and  control  documents;  distributing 
the  assessment  materials  to  the  field;  receiving,  tracking,  processing  and  editing  the  assessment  materials 
as  they  returned  from  the  field;  scoring  all  of  the  constructed-response  items  (in  conjunction  with  ETS); 
and  delivering  the  assessment  data  files  to  ETS  for  analysis  and  reporting. 

For  this  assessment,  NCS  was  charged  with  processing  and  scoring  the  largest  assessment  in  the 
history  of  NAEP  in  the  shortest  amount  of  time.  Further,  image  scanning  processes,  eliminating  almost  all 
paper  handling  during  scoring  and  improving  monitoring  and  reliability  scoring,  increased  to  nearly  twice 
that  of  the  1994  assessment.  Materials  management  and  distribution  of  over  one  quarter  of  a million 
science  kits,  receipt  control  processing  for  all  receipts  within  48  hours,  image  scanning  throughput 
increased  nearly  twice  that  of  1995,  and  professional  scoring  of  over  nine  million  constructed  responses 
highlight  the  challenges  met  by  NCS  for  the  1996  NAEP  national  and  state  assessments. 

NCS  processed  more  than  134,000  booklets  for  the  NAEP  1996  national  assessment,  as  shown  in 
Tables  6-1  and  6-2.  NCS  also  received  and  processed  a total  of  over  2,300  school  characteristics  and 
policies  questionnaires,  over  4,700  teacher  questionnaires,  and  over  16,274  SD/LEP  questionnaires  for 
the  three  grades,  as  shown  in  Table  6-3.  Table  6-4  lists  key  events  and  dates  in  the  NAEP  schedule. 

This  chapter  of  the  report  reviews  the  activities  conducted  by  NCS  for  the  NAEP  1996  main  and 
long-term  trend  assessments. 


^ Bradley  Thayer  was  the  NCS  project  manager  for  1996  NAEP,  Patrick  Bourgeacq  was  the  NCS  project  director  for  1996  NAEP 
scoring,  and  Timothy  Robinson  was  the  NCS  senior  processing  coordinator  for  1996  NAEP. 
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Table  6-1 

Processing  and  Scoring  Totals  for  the  1996  NAEP.  Assessment 


Booklets 

Processed 

Number  of 
Constructed- 
Response 
Items* 

Number  of 
Responses 
Scored^ 

Number  of 
Scorers  and 
• Team 
Leaders^ 

Length  of 
Training  and 
Scoring 

Fall  Trend 
ReadingAV  riting 

5,287 

22 

42,741 

7/1 

11/29/95-  1/5/96 

Fall  Trend 
Mathematics 

5,727 

28 

88,782 

11/0 

12/18/95  - 1/5/96 

Winter  Trend 
ReadingAV  riting 

4,988 

20 

34,341 

6/1 

1/30/96  - 4/3/96 

Winter  Trend 
Mathematics 

5,442 

29 

86,322 

9/1 

3/25/96  - 3/29/96 

Spring  Trend 
Reading/Writing 

4,669 

25 

43,094 

6)1 

4/10/96  - 5/24/96 

Spring  Trend 
Mathematics 

3,570 

29 

86,478 

5/1 

5/13/96  - 5/22/96 

Long-Term  Trend 
Writing  Holistic 

N/A 

6 

63,793 

56/7 

6/10/96  - 6/16/96 

Long-Term  Trend 
Writing  Mechanics 

N/A 

• 3 

2,329 

33/6 

7/15/96-7/31/96 

1990  Rescore 

4th  Grade  Mathematics 

749 

12 

8,988 

198/17 

3/13/96  - 5/6/96 

1990  Rescore 

8th  Grade  Mathematics 

730 

18 

13,374 

198/17 

3/13/96  - 5/6/96 

1990  Rescore 

12th  Grade  Mathematics 

725 

21 

15,225 

198/17 

3/13/96  - 5/6/96 

1992  Rescore 

4th  Grade  Mathematics 

2,498 

36 

27,031 

198/17 

3/13/96-5/6/96 

1992  Rescore 

8th  Grade  Mathematics 

2,498 

44 

33,077 

198  / 17 

3/13/96  - 5/6/96 

1992  Rescore 

12th  Grade  Mathematics 

2,498 

44 

32,387 

198/17 

3/13/96  - 5/6/96 

National  4th  Grade 
Mathematics  Spiral 

10,816 

64 

170,219 

198/17 

3/13/96  - 5/6/96 

National  4th  Grade 
Mathematics  Estimation 

2,128 

3 

7,756 

198/17 

3/13/96  - 5/6/96 

National  4th  Grade 
Mathematics  Theme 

4,038 

13 

29,750 

198/17 

3/13/96  - 5/6/96 

National  8th  Grade 
Mathematics  Spiral 

11,554 

69 

195,764 

198/17 

3/13/96  - 5/6/96 

' This  is  the  number  of  discrete  constructed-response  items  in  assessment  booklets. 

^ This  is  the  number  of  student  responses  to  the  constructed-response  items.  These  scored  responses  include  those  that  were 
rescored  for  reliability  estimation. 

^ Because  readers  scored  items  from  all  grades  and  all  types  of  booklets,  it  is  not  possible  to  break  the  numbers  down  by  how 
many  scored  each  classification  of  items. 

(continued) 


O 16 


ERIC 


140 


Table  6-1  (continued) 

Processing  and  Scoring  Totals  for  the  1996  NAEP  Assessment 


Booklets 

Processed 

Number  of 
Constructed- 
Response 
Items' 

Number  of 
Responses 
: Scored^ 

Number  of 
Scorers  and 
Team 
Leaders^ 

Length  of 
Training  and 
Scoring 

National  8th  Grade 
Mathematics  Estimation 

2,267 

6 

17,027 

198/17 

3/13/96  - 5/6/96 

National  8th  Grade 
Mathematics  Theme 

4,259 

13 

34,613 

198/17 

3/13/96-5/6/96 

National  8th  Grade 
Mathematics  Advanced 

2,382 

14 

41,693  - 

198/17 

3/13/96  - 5/6/96 

National  12th  Grade 
Mathematics  Spiral 

10,740 

73 

225,540 

198/17 

3/13/96  - 5/6/96 

National  12th  Grade 
Mathematics  Estimation 

1,883 

0 

0 

198/17 

3/13/96-5/6/96 

National  12th  Grade 
Mathematics  Theme 

3,892 

12 

29,210 

198/17 

3/13/96  - 5/6/96 

National  12th  Grade 
Mathematics  Advanced 

2,987 

15 

56,101 

198/17 

3/13/96  - 5/6/96 

Bilingual  4th  Grade 
Mathematics  National 

91 

10 

2,280 

0/4 

5/2/96  - 5/2/96 

Bilingual  8th  Grade 
Mathematics  National 

36 

12 

1,080 

0/4 

5/3/96  - 5/3/96 

National  4th  Grade 
Science  Spiral 

11,677 

94 

275,339 

306  / 24 

3/18/96-5/28/96 

National  8th  Grade 
Science  Spiral 

12,079 

125 

322,261 

306  / 24 

3/18/96  - 6/7/96 

National  12th  Grade 
Science  Spiral 

11,579 

120 

342,104 

306  / 24 

3/18/96  - 6/2/96 

National  12th  Grade 
Science  Advanced 

2,449 

36 

110,207 

306  / 24 

3/18/96  - 6/2/96 

' This  is  the  number  of  discrete  constructed-response  items  in  assessment  booklets. 

^ This  is  the  number  of  student  responses  to  the  constructed-response  items.  These  scored  responses  include  those  that  were 


rescored  for  reliability  estimation. 

’ Because  readers  scored  items  from  all  grades  and  all  types  of  booklets,  it  is  not  possible  to  break  the  numbers  down  by  how 
many  scored  each  classification  of  items. 
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Table  6-2 

Student  Participation  and  Session  Information  for  the  1996  NAEP  Assessment 


Number 

of 

Sessions 

Number  of 
Booklets  for 
Assessed 
Students 

Number  of 
Booklets 
for  Absent 
Students 

Number  of 
Booklets  for 
Excluded 
Student 

Number 

of 

Scanned 

Sheets 

Long-Term  Trend 

Fall 

639 

11,150 

981 

974 

288,624 

Winter 

623 

10,406 

486 

1,172 

215,679 

Spring 

539 

8,209 

1,560 

758 

243,452 

Main 

Grade  4 total 

1,693 

28,531 

1,354 

2,293 

711,291 

Mathematics 

982 

16,953 

796 

1,198 

418,509 

Science 

711 

11,578 

558 

1,095 

292,782 

Grade  8 total 

1,698 

32,339 

2,359 

1,823 

922,892 

Mathematics 

819 

17,992 

1,338 

1,062 

509,064 

Advanced  Mathematics 

345 

2,375 

94 

4 

73,268 

Science 

534 

11,972 

927 

757 

340,560 

Grade  12  total 

2,196 

33,306 

8,266 

1,704 

1,143,332 

Mathematics 

848 

16,424 

4,103 

969 

536,256 

Advanced  Mathematics 

380 

2,965 

456 

11 

99,112 

Science 

613 

11,486 

3,332 

715 

387,504 

Advanced  Science 

355 

2,431 

375 

9 

120,460 

Other 

Rosters 

Administration  Schedules 

23,535 

24.575 

Rescore  Mathematics  1992 

Grade  4 

Rescore  Mathematics  1992 
Grade  8 

Rescore  Mathematics  1990 

45,286 

61,311 

46,507 

0'8 
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Table  6-3 

Questionnaire  Totals  for  the  1996  NAEP  Assessment 


Expected 

Received 

Percent 

Main  Assessment 
Grade  4 

SD/LEP  Questionnaire 

5,116 

4,885 

95% 

School  Characteristics  Questionnaire 

597 

577 

97% 

Grade  8 

SD/LEP  Questionnaire 

5,048 

4,770 

94% 

School  Characteristics  Questionnaire 

580 

554 

96% 

Grade  12  ^ 

SD/LEP  Questionnaire 

4,147 

3,806 

92% 

School  Characteristics  Questionnaire 

582 

546 

94% 

Long-Term  Trend 
Fall 

Excluded  Students  Questionnaire 

979 

947 

97% 

School  Characteristics  Questionnaire 

239 

224 

94% 

Winter  

Excluded  Students  Questionnaire 

1,187 

1,145 

96% 

School  Characteristics  Questionnaire 

247 

235 

95% 

Spring 

Excluded  Students  Questionnaire 

768 

721 

94% 

School  Characteristics  Questionnaire 

Main  Teacher  Questionnaires 

186 

173 

93% 

Grade  4 

Mathematics/Science 

1,599 

1,609 

101% 

Grade  8 

Mathematics 

1,401 

1,359 

97% 

Science 

1,310 

1,270 

97% 

Grade  12  

Advanced  Mathematics 

476 

487 

1U2% 

Table  6-4 

NCS  Schedule  for  the  1996  NAEP  Assessment 


Task 

Planned 

Start 

Planned 

Finish 

Actual 

Start 

Actual 

Finish 

FALL  LONG-TERM  TREND 

Printing 

6/15/95 

9/19/95 

6/15/95 

9/19/95 

Pre-packaging  (barcoding,  spiraling,  and  quality 

control) 

8/14/95 

9/7/95 

8/14/95 

9/7/95 

Session  file  to  NCS  from  Westat 

8/25/95 

8/25/95 

8/25/95 

8/25/95 

Westat  supervisor  training 

8/27/95 

8/30/95 

8/27/95 

8/30/95 

Print  Administration  Schedule 

9/1/95 

9/5/95 

9/1/95 

9/1/95 

Supervisor  add  file  from  Westat 

9/5/95 

9/5/95 

9/5/95 

9/5/95 

Final  packing  specifications  to  packaging 

9/5/95 

9/5/95 

9/5/95 

9/5/95 

Packing  list  and  labels  to  packaging 
Grade  8 school  characteristics  and  policies 

9/5/95 

9/6/95 

9/6/95 

9/6/95 

questionnaires  arrive 

9/7/95 

9/7/95 

9/7/95 

9/19/95 

Ship  Administration  Schedules 

9/7/95 

9/7/95 

9/7/95 

9/7/95 

Information  to  Key  Entry  for  screen  set  up 

9/11/95 

9/11/95 

9/11/95 

9/1 1/95 

Bulk/session  packaging 

9/11/95 

9/18/95 

9/11/95 

9/15/95 

Packaging  visit  by  ETS 

9/13/95 

9/14/95 

9/14/95 

9/14/95 

Ship  materials  to  supervisors 

9/18/95 

9/18/95 

9/14/95 

9/14/95 

Materials  due  to  supervisors 

9/22/95 

9/22/95 

9/22/95 

9/22/95 

Processing  specifications  to  Operations  Department 

10/2/95 

10/2/95 

10/2/95 

10/2/95 

Processing  kick-off  meeting 

10/3/95 

10/3/95 

10/3/95 

10/3/95 

Requisitions  for  table  leaders  to  HR 

10/9/95 

10/9/95 

10/9/95 

10/9/95 

Requisitions  for  scorers  to  HR 

10/9/95 

10/9/95 

10/9/95 

10/9/95 

Photocopy  training  materials 

10/9/95 

10/13/95 

10/9/95 

10/13/95 

Test  administration 

10/9/95 

12/19/95 

10/10/95 

12/19/95 

Blue  dot 

10/10/95 

10/20/95 

10/20/95 

10/20/95 

Receiving 

10/10/95 

12/23/95 

10/11/95 

1/15/96 

HR  extends  offer  to  table  leaders 

10/13/95 

10/13/95 

10/13/95 

10/13/95 

HR  extends  offers  to  scorers 

10/16/95 

10/16/95 

10/16/95 

10/16/95 

General  - Network  Meeting 

10/20/95 

10/20/95 

10/20/95 

10/20/95 

Processing 

10/20/95 

12/28/95 

10/16/95 

1/9/96 

Scoring  training  preparation 

10/9/95 

11/17/95 

10/9/95 

10/17/95 

Scoring  training  - writing 

11/29/95 

11/30/95 

11/29/95 

11/30/95 

Scoring  training  - reading 

12/4/95 

12/5/95 

12/4/95 

12/5/95 

Scoring  reading/writing 

11/29/95 

1/5/96 

11/29/95 

1/5/96 

Weights  file  shipped 

1/12/96 

1/15/96 

1/15/96 

1/15/96 

Tape  delivered 

1/12/96 

1/15/96 

1/25/96 

1/25/96 

WINTER  LONG-TERM  TREND 

Printing 

6/15/95 

9/19/95 

6/15/95 

9/19/95 

Pre-packaging  (barcoding,  spiraling) 

9/29/95 

10/10/95 

9/6/95 

9/14/95 

Bundle  sheets  delivered  to  packaging 

10/25/95 

10/25/95 

9/1/95 

9/1/95 

All  bundles  through  clean  quality  control 

11/3/95 

1 1/3/95 

9/1/95 

9/1/95 

(continued) 


Table  6-4  (continued) 

NCS  Schedule  for  the  1996  NAEP  Assessment 


Planned 

Planned 

Actual 

Actual 

Task 

Start 

Finish 

Start 

Finish 

WINTER  LONG-TERM  TREND  (continued) 

Final  packing  specifications  to  packaging 

11/17/95 

11/17/95 

11/16/95 

11/16/95 

HR  extends  offers  to  scorers 

1 1/20/95 

1 1/22/95 

1 1/20/95 

11/22/95 

Session  data  file  to  NCS  from  Westat 

11/22/95 

11/22/95 

11/22/95 

11/22/95 

Administration  Schedule  address  file  from  Westat 

11/28/95 

11/28/95 

1 1/22/95 

1 1/22/95 

Print  Administration  Schedules 

12/1/95 

12/1/95 

1 1/30/95 

11/30/95 

Bulk/session  address  file  from  Westat 

12/1/95 

12/1/95 

11/29/95 

11/29/95 

Ship  Administration  Schedules 

12/4/95 

12/4/95 

12/1/95 

12/1/95 

Packing  list  and  labels  to  packaging 

12/4/95 

12/5/95 

12/1/95 

12/1/95 

Final  packaging 

12/6/95 

12/8/95 

12/6/95 

12/7/95  • 

Ship  session  materials 

12/6/95 

12/8/95 

12/6/95 

12/7/95 

Processing  specifications  to  Operations 

12/18/95 

12/18/95 

12/18/95 

12/18/95 

Requisitions  for  scorers  to  HR 

12/20/95 

12/20/95 

12/20/95 

12/20/95 

Materials  due  to  supervisors 

12/22/95 

12/22/95 

12/15/95 

12/18/95 

Test  administration 

1/3/96 

3/15/96 

1/2/96 

3/15/96 

Blue  dot  (s) 

1/4/96 

1/8/96 

1/15/96 

1/22/96 

Receiving 

1/4/96 

3/20/96 

1/9/96 

3/20/96 

Image  definition  ready 

1/8/96 

1/8/96 

1/5/96 

1/5/96 

Processing 

1/8/96 

3/22/96 

1/8/96 

3/22/96 

Photocopy  training  materials 

1/9/96 

1/12/96 

1/9/95 

10/13/95 

Image  test  data  ready 

1/9/96 

1/12/96 

1/9/96 

1/12/96 

Image  application  ready 

1/11/96 

1/11/96 

1/11/96 

1/11/96 

Scoring  training  preparation 

1/29/96 

1/31/96 

1/29/96 

1/29/96 

Scoring  training  - writing 

1/31/96 

2/2/96 

1/30/96 

1/31/96 

Scoring  training  - reading 

1/31/96 

2/2/96 

2/6/96 

2/6/96 

Scoring  reading/writing 

2/5/96 

4/5/96 

2/7/96 

4/2/96 

Requisitions  for  mathematics  scorers  to  HR 

3/1/96 

3/1/96 

3/11/96 

3/1 1/96 

Scoring  training  - mathematics 

3/25/96 

3/25/96 

3/25/96 

3/25/96 

Scoring  - mathematics 

3/25/96 

4/5/96 

3/25/96 

3/29/96 

Tape  delivered 

4/15/96 

4/15/96 

4/12/96 

4/12/96 

Weights  file  shipped 

4/15/96 

4/15/96 

4/12/96 

4/12/96 

SPRING  LONG-TERM  TREND 

Printing 

6/15/95 

9/19/95 

6/15/95 

9/19/95 

Pre-packaging  (barcoding,  spiraling) 

7/21/95 

9/25/95 

7/21/95 

9/25/95 

HR  extends  offers  to  table  leaders 

10/13/95 

10/13/95 

10/13/95 

10/13/95 

HR  extends  offers  to  scorers 

1 1/20/95 

1 1/22/95 

1 1/20/95 

1 1/22/95 

Session  file  to  NCS  from  Westat 

2/5/96 

2/5/96 

2/5/96 

2/5/96 

Bulk/session  address  file  from  Westat 

2/8/96 

2/8/96 

2/5/96 

2/5/96 

Administration  Schedule  address  file  from  Westat 

2/8/96 

2/8/96 

2/7/96 

. 2/7/96 

Packing  list  and  labels  to  packaging 

2/8/96 

2/8/96 

2/9/96 

2/9/96 

Final  packing  specifications  to  packaging 

2/8/96 

2/8/96 

2/12/96 

2/12/96 

(continued) 


Table  6-4  (continued) 

NCS  Schedule  for  the  1996  NAEP  Assessment 


Task 

Planned 

Start 

Planned 

Finish 

Actual 

Start 

Actual 

Finish 

SPRING  LONG-TERM  TREND  (continued) 
Print  Administration  Schedules 

2/12/96 

2/12/96 

2/9/96 

2/9/96 

Ship  Administration  Schedules 

2/16/96 

2/16/96 

2/13/96 

2/13/96 

Package/ship  session  materials 

2/23/96 

2/23/96 

2/23/96 

2/23/96 

Material  due  to  supervisors 

2/28/96 

2/28/96 

3/1/96 

3/1/96 

Processing  specifications  to  Operations 

3/8/96 

3/8/96 

3/8/96 

3/8/96 

Rescore  booklets  delivered  to  PSC 

3/8/96 

3/8/96 

3/8/96 

3/8/96 

Photocopy  training  materials 

3/9/96 

3/12/96 

10/9/95 

10/13/95 

Test  administration 

3/11/96 

5/10/96 

3/1 1/96 

5/17/96 

PSC  approval  of  scoring  sheets 

3/15/96 

3/15/96 

3/13/96 

3/13/96 

Blue  dot  (s) 

3/18/96 

3/20/96 

4/2/96 

4/19/96 

Processing 

3/21/96 

5/17/96 

4/19/96 

5/17/96 

Requisitions  for  mathematics  scorers  to  HR 

4/1/96 

4/1/96 

4/1/96 

4/1/96 

Scoring  training  preparation 

4/8/96 

4/9/96 

4/3/96 

4/3/96 

Scoring  training  - writing 

4/10/96 

4/11/96 

4/10/96 

4/1 1/96 

Scoring  reading/writing 

4/10/96 

5/24/96 

4/11/96 

4/11/96 

Scoring  training  - reading 

4/11/96 

4/11/96 

4/11/96 

4/11/96 

Scoring  training  - mathematics 

5/13/96 

5/13/96 

5/13/96 

5/13/96 

Scoring  mathematics 

5/13/96 

5/24/96 

5/13/96 

5/24/96 

Project  through  clean  post 

5/17/96 

5/17/96 

5/17/96 

5/17/96 

Tape  delivered 

5/31/96 

5/31/96 

6/6/96 

6/6/96 

Weights  file  shipped 

5/31/96 

5/31/96 

6/6/96 

6/6/96 

WRITING  LONG-TERM  TREND  - 
HOLISTIC  SCORING 

Requisition  for  scorers  to  HR 

5/15/96 

5/15/96 

5/15/96 

5/15/96 

Requisition  for  table  leaders  to  HR 

5/15/96 

5/15/96 

5/15/96 

5/15/96 

PSC  approves  score  sheet 

5/15/96 

5/15/96 

5/15/96 

5/15/96 

HR  makes  offers  to  table  leaders 

5/20/96 

5/31/96 

5/28/96 

6/4/96 

HR  makes  offers  to  scorers 

5/20/96 

5/31/96 

5/28/96 

6/7/96 

Samples  drawn 

5/24/96 

5/24/96 

5/20/96 

5/24/96 

Scoring  preparation  with  J.  Kennedy 

6/4/96 

6/7/96 

6/5/96 

6/7/96 

Training  and  Scoring 

6/10/96 

6/14/96 

6/10/96 

6/16/96 

Data  tape  delivered 

7/1/96 

7/1/96 

6/27/96 

6/27/96 

WRITING  LONG-TERM  TREND  - 
MECHANICS  SCORING 
Requisition  for  scorers  to  HR 

6/14/96 

6/14/96 

7/9/96 

7/9/96 

Requisition  for  table  leaders  to  HR 

6/14/96 

6/14/96 

7/9/96 

7/9/96 

HR  makes  offers  to  scorers 

6/24/96 

6/28/96 

7/9/96 

7/12/96 

HR  makes  offers  to  table  leaders 

6/24/96 

6/28/96 

7/9/96 

7/10/96 

Samples  drawn 

6/24/96 

6/24/96 

6/24/96 

6/24/96 

(continued) 
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Table  6-4  (continued) 

NCS  Schedule  for  the  1996  NAEP  Assessment 


Task 

Planned 

Start 

Planned 

Finish 

Actual 

Start 

Actual 

Finish 

WRITING  LONG-TERM  TREND  - 
MECHANICS  SCORING  (continued) 
Responses  copied  for  scoring 

6/25/96 

7/12/96 

6/25/96 

7/12/96 

Training  and  scoring 

7/15/96 

7/26/96 

7/15/96 

7/31/96 

Transcribe  and  proofread  essays  and  scores 

7/18/96 

7/31/96 

7/18/96 

7/31/96 

Data  tape  delivered 

8/9/96 

8/9/96 

8/7/96 

8/7/96 

MAIN  ASSESSMENT 
Printing 

9/2/95 

12/11/95 

9/2/95 

12/11/95 

Administration  Schedule  approved 

9/15/95 

9/15/95 

9/21/95 

9/21/95 

Pre-packing  specifications  to  packaging 

10/2/95 

10/2/95 

10/2/95 

10/2/95 

Grade  8 teacher  questionnaire  roster  delivered  to  NCS 

10/12/95 

10/12/95 

10/16/95 

10/16/95 

Grade  4 mathematics/science  teacher  questionnaire 
roster  to  NCS 

10/12/95 

10/12/95 

10/17/95 

10/17/95 

PSC  obtains  copies  of  final  blocks 

10/16/95 

10/16/95 

10/16/95 

10/16/95 

Administration  Schedule  delivered  to  NCS 

10/18/95 

10/18/95 

10/23/95 

10/23/95 

Grade  1 2 teacher  questionnaire  roster  delivered  to 
NCS 

10/20/95 

10/20/95 

10/16/95 

10/16/95 

Grade  8 school  characteristics  and  policies 
questionnaires  at  NCS 

10/20/95 

10/20/95 

10/23/95 

10/23/95 

SD/LEP  roster  delivered  to  NCS 

10/20/95 

10/20/95 

10/24/95 

10/24/95 

Grade  4 school  characteristics  and  policies 
questionnaires  at  NCS 

10/20/95 

10/20/95 

10/25/95 

10/25/95 

Grade  12  school  characteristics  and  policies 
questionnaires  at  NCS 

10/20/95 

10/23/95 

10/23/95 

10/23/95 

Grade  8 mathematics  spiral  material  at  NCS 

10/23/95 

11/2/95 

10/18/95 

1 1/3/95 

Pre-packaging  begins 

10/23/95 

12/20/95 

10/16/95 

12/1/95 

Grade  4 mathematics  spiral  material  at  NCS 

10/26/95 

11/1/95 

11/1/95 

11/1/95 

Grade  8 mathematics  teacher  questionnaire  at  NCS 

10/30/95 

10/30/95 

10/25/95 

10/25/95 

Grade  8 science  teacher  questionnaire  at  NCS 

10/30/95 

10/30/95 

11/1/95 

11/1/95 

Final  valid  score  range  for  each  item 

11/1/95 

11/1/95 

10/24/95 

11/17/95 

PSC  obtains  rubrics  from  ETS 

11/1/95 

11/1/95 

10/24/95 

1 1/17/95 

ETS/PSC  define  non-scorable  codes 

11/1/95 

11/1/95 

11/1/95 

11/1/95 

NCS/ETS  meet  to  review  items  and  schedule 

11/2/95 

11/3/95 

11/2/95 

11/3/95 

Grade  4 mathematics/science  teacher  questionnaire 
delivered  to  NCS 

11/3/95 

11/3/95 

11/2/95 

11/2/95 

Grade  8 science  spiral  material  at  NCS 

11/6/95 

11/13/95 

11/13/95 

1 1/21/95 

Grade  12  mathematics  spiral  material  at  NCS 

11/14/95 

11/21/95 

11/21/95 

12/1/95 

General  - sub-contractor’s  meeting 

11/16/95 

11/17/95 

11/16/95 

1 1/17/95 

SD/LEP  questionnaire  delivered  to  NCS 

11/22/95 

11/22/95 

12/5/95 

12/5/95 

Grade  4 science  spiral  material  at  NCS 

11/22/95 

11/30/95 

11/21/95 

12/1/95 

(continued) 
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Table  6-4  (continued) 

NCS  Schedule  for  the  1996  NAEP  Assessment 


Planned  Planned  Actual  Actual 
Start  . Finish Start  Finish 


MAIN  ASSESSMENT  (continued) 

All  materials  at  NCS  for  packaging 

Grade  12  science  spiral  material  at  NCS 

NCS  receive  95%  session  data  from  Westat 

Westat  training  for  main  supervisors 

Westat  send  Administration  Schedule  home  address 

file 

Westat  send  NCS  Wave  1 address  file 

Print  Administration  Schedule 

WAVE  1 packing  list  and  labels  to  packaging 

Format/content  Interrater  Agreement  Report 

PSC  obtains  sample  booklets 

Purpose/use  of  T-Test  and  bridge  reliability 

PSC  submit  requisition  for  mathematics  and  science 

scorers 

PSC  submit  requisition  for  mathematics  and  science 
team  leaders 

PSC  submit  requisition  for  mathematics  and  science 
table  leaders 

Ship  Administration  Schedule,  teacher  questionnaire, 

SD/LEP  questionnaires,  rosters 

Ship  bulkAVave  1 material 

Bulk/Wave  1 materials  due  to  supervisors 

Receiving 

Test  administration 

Blue  dot  Grades  4,  8 and  12  mathematics 
Blue  dot  Grades  4,  8,  and  12  science 
Blue  dot  advanced  science 

Blue  dot  Grade  12  estimation,  theme,  and  advanced 

mathematics 

Processing 

Blue  dot  SD/LEP  questionnaires 
Assignments  of  mathematics  items  to  teams 
Scoring  calendar  item-by-item  for  mathematics 
Wave  2 addresses  from  Westat 
Packaging  Wave  2 materials  (2  shifts) 

Wave  2 packing  and  mailing  labels  to  packaging 
Blue  dot  school  characteristics  and  policies 
questionnaires  and  teacher  questionnaires 
Blue  dot  short-term  rescore 

Assignment  of  science  items  to  teams  


11/29/95 

12/1/95 

12/1/95 

12/15/95 

12/1/95 

12/11/95, 

12/1/95 

12/4/95 

12/4/95 

12/4/95 

12/6/95 

12/6/95 

12/9/95 

12/13/95 

12/9/95 

12/13/95 

12/11/95 

12/11/95 

12/11/95 

12/11/95 

12/11/95 

12/11/95 

12/11/95 

12/11/95 

12/11/95 

12/13/95 

12/13/95 

12/13/95 

12/13/95 

12/13/95 

12/14/95 

12/14/95 

12/15/95 

12/15/95 

1 1/1/95 

12/15/95 

12/15/95 

12/15/95 

11/2/95 

11/13/95 

12/15/95 

12/15/95 

1 1/2/95 

11/3/95 

12/15/95 

12/15/95 

1/6/96 

1/16/96 

12/15/95 

12/15/95 

1/16/96 

1/16/96 

12/15/95 

12/15/95 

1/30/96 

1/30/96 

12/15/95 

12/18/95 

12/14/95 

12/14/95 

12/20/95 

12/20/95 

12/15/95 

12/18/95 

12/26/95 

12/26/95 

12/26/95 

12/29/95 

1/5/96 

4/8/96 

1/5/96 

4/8/96 

1/3/96 

4/5/96 

1/3/96 

4/5/96 

1/4/96 

1/7/96 

1/15/96 

1/25/96 

1/4/96 

1/7/96 

1/15/96 

1/25/96 

1/4/96 

1/7/96 

1/15/96 

1/26/96 

1/4/96 

1/7/96 

1/15/96 

1/31/96 

1/8/96 

4/14/96 

1/25/96 

4/29/96 

1/9/96 

1/11/96 

2/20/96 

2/27/96 

1/15/96 

1/15/96 

1/2/96 

2/23/96 

1/15/96 

1/15/96 

1/2/96 

2/15/96 

1/17/96 

1/17/96 

1/17/96 

1/17/96 

1/18/96 

1/26/96 

1/19/96 

1/23/96 

1/17/96 

1/19/96 

1/19/96 

1/19/96 

1/22/96 

1/25/96 

2/9/96 

2/26/96 

1/25/96 

1/25/96 

3/5/96 

3/11/96 

2/1/96 

2/1/96 

1/15/96 

5/31/96 
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(continued) 


Table  6-4  (continued) 

NCS  Schedule  for  the  1996  NAEP  Assessment 

Planned  Planned  Actual  Actual 
Task  Start Finish  Start  Finish 


MAIN  ASSESSMENT  (continued) 

Mathematics  and  science  table  leaders  hired 
Plan  for  staff  range  finding 
PSC  selects  science  team  leaders 
PSC  selects  mathematics  team  leaders 
Blue  dot  rosters  (all  types) 

Wave  2 materials  due  to  supervisors 
Scoring  calendar  item  by  item  (science) 

Wave  3 addresses  from  Westat 

Segment  1 day  25%  scorers  hired 

Wave  3 packing  list/mailing  labels  to  packaging 

Packaging/ship  Wave  3 materials 

Pre-range  finding  paper  selection  - mathematics 

Pre-range  finding  paper  selection  - science 

Segment  1 day  50%  scorers  hired 

Segment  1 day  75%  scorers  hired 

Wave  3 materials  due  in  supervisors 

PSC  selects  mathematics  table  leaders 

PSC  selects  science  table  leaders 

Segment  2 day  25%  scorers  hired 

Segment  3 day  25%  scorers  hired 

Segment  3 evening  25%  scorers  hired 

Segment  2 evening  25%  scorers  hired 

"Segment  1 day  100%  scorers  hired 

Table  leaders  for  mathematics  hired 

Segment  3 day  50%  scorers  hired 

Segment  2 day  50%  scorers  hired 

Segment  2 evening  50%  scorers  hired 

Segment  3 evening  50%  scorers  hired 

Mathematics  and  science  scorers  assigned  to  teams 

Train/score  mathematics  - Segment  l(day  shift) 

Segment  2 day  75%  scorers  hired 

Segment  3 day  75%  scorers  hired 

Segment  2 evening  75%  scorers  hired 

Segment  3 evening  75%  scorers  hired 

Train/score  science  - Segment  1 (day  shift) 

Segment  2 evening  100%  scorers  hired 

Segment  3 evening  100%  scorers  hired 

Segment  2 day  100%  scorers  hired 

Segment  3 day  100%  scorers  hired 

Table  leaders  for  science  hired 


2/1/96 

2/1/96 

1/15/96 

2/28/96 

2/1/96 

2/1/96 

1/15/96 

2/1/96 

2/1/96 

2/1/96 

1/15/96 

1/16/96 

2/1/96 

2/1/96 

1/16/96 

1/16/96 

2/1/96 

2/1/96 

1/26/96 

2/12/96 

2/2/96 

2/2/96 

2/1/96 

2/2/96 

2/3/96 

2/6/96 

1/15/96 

5/31/96 

2/7/96 

2/7/96 

2/7/96 

2/7/96 

2/12/96 

2/12/96 

2/7/96 

2/7/96 

2/12/96 

2/12/96 

2/9/96 

2/13/96 

2/12/96 

2/19/96 

2/19/96 

2/20/96 

2/12/96 

3/8/96 

2/5/96 

3/8/96 

2/12/96 

3/15/96 

2/5/96 

3/8/96 

2/19/96 

2/19/96 

2/9/96 

2/9/96 

2/26/96 

2/26/96 

2/15/96 

2/15/96 

2/26/96 

2/26/96 

2/26/96 

2/27/96 

. 3/1/96 

3/1/96 

2/1/96 

2/28/96 

3/1/96 

3/1/96 

2/1/96 

2/28/96 

3/1/96 

3/1/96 

2/8/96 

2/8/96 

3/1/96 

3/1/96 

2/8/96 

2/8/96 

3/1/96 

3/1/96 

2/20/96 

2/20/96 

3/1/96 

3/1/96 

2/22/96 

2/22/96 

3/4/96 

3/4/96 

3/6/96 

3/6/96 

3/8/96 

3/8/96 

2/1/96 

3/22/96 

3/8/96 

3/8/96 

2/14/96 

2/14/96 

3/8/96 

3/8/96 

2/15/96 

2/15/96 

3/8/96 

3/8/96 

3/6/96 

3/6/96 

3/8/96 

3/8/96 

3/6/96 

3/6/96 

3/11/96 

3/11/96 

3/11/96 

3/11/96 

3/13/96 

4/3/96 

3/13/96 

4/5/96 

3/15/96 

3/15/96 

3/5/96 

3/14/96 

3/15/96 

3/15/96 

3/6/96 

3/6/96 

3/15/96 

3/15/96 

3/14/96 

. 3/14/96 

3/15/96 

3/15/96 

3/14/96 

3/14/96 

3/18/96 

4/5/96 

3/18/96 

4/5/96 

3/22/96 

3/22/96 

3/25/96 

3/25/96 

3/22/96 

3/22/96 

3/25/96 

3/25/96 

3/22/96 

3/22/96 

4/1/96 

4/1/96 

3/22/96 

3/22/96 

4/1/96 

4/1/96 

4/5/96 

4/5/96 

2/12/96 

2/28/96 

(continued) 
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Table  6-4  (continued) 

NCS  Schedule  for  the  1996  NAEP  Assessment 


Task 

Planned 

Start 

Planned 

Finish 

Actual 

Start 

Actual 

Finish 

MAIN  ASSESSMENT  (continued) 

Train/score  mathematics  - Segment  2 (evening  shift) 

4/8/96 

5/2/96 

4/8/96 

5/2/96 

Train/score  science  - Segment  2 (evening  shift) 

4/8/96 

5/2/96 

4/8/96 

5/2/96 

Train/score  mathematics  - Segment  2 (day  shift) 

4/8/96 

5/3/96 

4/8/96 

5/6/96 

Train/score  science  - Segment  2 (day  shift) 

4/8/96 

5/3/96 

4/8/96 

5/3/96 

Mathematics  through  clean  post 

4/14/96 

4/14/96 

4/14/96 

4/29/96 

Science  through  clean  post  edit 

4/28/96 

4/28/96 

4/26/96 

4/26/96 

Bilingual  mathematics  scoring 

5/2/96 

5/3/96 

5/2/96 

5/3/96 

Grade  8 mathematics  weights 

5/4/96 

5/6/96 

5/3/96 

5/3/96 

Grade  4 mathematics  weights 

5/4/96 

5/6/96 

5/9/96 

5/9/96 

Grade  4 science  data  tape  sent  to  ETS 

5/31/96 

5/31/96 

5/30/96 

5/30/96 

School  characteristics  and  policies  questionnaires  data 
tape  shipped  to  ETS 

7/11/96 

7/12/96 

7/11/96 

7/11/96 

Teacher  questionnaire  data  tape  shipped  to  ETS 

7/18/96 

7/19/96 

7/19/96 

7/24/96 

SD/LEP  questionnaire  data  shipped  to  ETS 

7/26/96 

7/29/96 

8/2/96 

8/2/96 

Grade  12  science  data  tape  sent  to  ETS 

6/7/96 

6/7/96 

6/4/96 

6/4/96 

Grade  12  advanced  science  data  tape  sent  to  ETS 

6/7/96 

6/7/96 

6/10/96 

6/10/96 

Grade  8 science  data  tape  sent  to  ETS 

5/31/96 

5/31/96 

6/26/96 

6/26/96 

6.1.1  Innovations  for  1996 


Much  of  the  information  necessary  for  documentation  of  accurate  sampling  and  for  calculating 
sampling  weights  is  collected  on  the  Administration  Schedules  that,  until  1993,  were  painstakingly  filled 
out  by  hand  by  Westat  administrative  personnel.  In  1994,  for  the  first  time,  much  of  the  work  was 
computerized — booklets  were  preassigned  and  booklet  ID  numbers  were  preprinted  on  the  Administration 
Schedule.  When  Westat  personnel  received  the  documents,  they  filled  in  only  the  “exception” 
information.  This  new  method  also  permitted  computerized  updating  of  information  when  the 
Administration  Schedules  were  received  at  NCS,  eliminating  the  need  to  sort  and  track  thousands  of 
pieces  of  paper  through  the  processing  stream. 

The  introduction  of  image  processing  and  image  scoring  further  enhanced  the  work  of  NAEP. 
Image  processing  and  scoring  were  successfully  piloted  in  a side-by-side  study  conducted  during  the  1993 
NAEP  field  test,  and  so  became  the  primary  processing  and  scoring  methods  for  the  1994  and  1996 
assessments.  Image  processing  allowed  the  automatic  collection  of  handwritten  demographic  data  from 
the  administrative  schedules  and  the  student  test  booklet  covers  through  intelligent  character  recognition 
(ICR).  This  service  was  a benefit  to  the  jurisdictions  participating  in  NAEP  because  they  were  able  to 
write  rather  than  grid  certain  information — a reduction  of  burden  on  the  schools.  Image  processing  also 
made  image  scoring  possible,  eliminating  much  of  the  time  spent  moving  paper  as  part  of  the  scoring 
process.  The  images  of  student  responses  to  be  scored  were  transmitted  electronically  to  the  scoring 
center,  located  at  a separate  facility  from  where  the  materials  were  processed.  This  process  enhanced  the 
reliability  and  monitoring  of  scoring  and  allowed  both  NCS  and  ETS  to  focus  attention  on  the  intellectual 
process  of  scoring  student  responses. 
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6.2  PRINTING 


For  the  1996  assessment,  255  unique  documents  were  designed.  NCS  printed  more  than 
1,900,000  booklets  and  forms,  totaling  over  58  million  pages.  Printing  preparations  began  with  the  design 
of  the  booklet  covers  in  June  1995.  This  was  a collaborative  effort  involving  staff  from  ETS,  Westat  and 
NCS.  Since  the  goal  was  to  design  one  format  for  use  with  all  of  the  booklets,  necessary  data  elements  to 
be  collected  for  the  different  assessment  types  had  to  be  agreed  upon.  After  various  iterations,  the  cover 
design  was  finalized. 

In  a similar  collaboration  with  ETS  and  Westat,  NCS  prepared  administration  schedules  and 
questionnaire  rosters.  The  camera-ready  copies  for  these  documents  were  created  and  edited  using  NCS 
Design  Expert™  software. 

Printing  of  the  NAEP  documents  began  with  the  documents  for  the  long-term  trend  assessments. 
These  included  the  26  long-term  trend  assessment  booklets,  the  Administration  Schedule,  the  excluded 
student  questionnaire,  three  school  characteristics  and  policies  questionnaires,  and  the  roster  of 
questionnaires.  All  materials  for  the  long-term  trend  assessments  were  printed  by  September  19,  1995. 

The  printing  of  assessment  booklets,  questionnaires  and  tracking  forms  for  the  main  and  state  assessments 
followed.  Printing  of  these  documents  was  complete  by  December  1 1,  1995. 

Details  of  the  printing  procedures  are  given  in  the  Report  of  Processing  and  Professional  Scoring 
Activities  (National  Computer  Systems,  1996). 

6.3  PACKAGING  AND  SHIPPING 

The  distribution  effort  for  the  1996  NAEP  assessment  involved  packaging  and  mailing  documents 
and  associated  forms  and  materials  to  the  Westat  supervisors  for  the  main  and  long-term  trend 
assessments.  The  NAEP  Materials  Distribution  System  (MDS),  initially  developed  by  NCS  in  1990  to 
control  shipments  to  the  schools  and  supervisors,  was  utilized  again  in  1996.  Files  in  the  MDS  system 
contained  the  names  and  addresses  for  shipment  of  materials,  scheduled  assessment  dates,  and  a listing  of 
all  materials  available  for  use  by  a participant  in  a particular  subject  area.  Changes  to  any  of  this 
information  were  made  directly  in  the  MDS  file  either  manually  or  via  file  updates  provided  by  Westat. 
Details  of  the  accountability  system  and  on-line  bundle  assignment  and  distribution  system  utilized  for 
NAEP  are  given  in  the  Report  of  Processing  and  Professional  Scoring  Activities. 

The  bar  code  technology  introduced  by  NCS  in  the  1990  assessments  continued  to  be  utilized  in 
document  control.  To  identify  each  document,  NCS  utilized  a unique  ten-digit  numbering  system  that 
consisted  of  the  three-digit  booklet  number  or  form  type,  a six-digit  sequential  number,  and  a check  digit. 
Each  form  was  assigned  a range  of  ID  numbers.  Bar  codes  reflecting  this  ID  number  were  applied  to  the 
front  cover  of  each  document  by  NCS  bar  code  processes  and  high-speed  inkjet  printers. 

Once  all  booklets  from  a subject  area  were  bar  coded,  they  were  spiraled  and  bundled  into  groups 
of  1 1 documents.  For  main  samples  in  mathematics  and  science  (done  concurrently  with  the  State 
Assessment  samples  in  mathematics  and  science),  NCS  spiraled  the  booklets  according  to  the  pattern 
dictated  by  ETS  in  the  bundle  maps.  Booklets  were  spiraled  in  such  a manner  that  each  booklet  appeared 
in  the  first  position  in  a bundle  approximately  the  same  number  of  times  and  the  booklets  were  evenly 
distributed  across  the  bundles.  This  assured  that  sample  sizes  of  individual  booklet  types  would  not  be 
jeopardized  if  entire  bundles  were  not  used.  Since  the  mathematics  and  science  estimation  and  advanced 
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booklet  bundles  contained  only  one  booklet  type,  these-were  bundled  into  groups  of  1 1.  The  mathematics 
bilingual  booklets  were  bundled  in  groups  of  three. 

Initially  5,161  individual  sessions  were  shipped  for  the  1996  NAEP  assessments.  Approximately 
600  additional  shipments  of  booklets  and  miscellaneous  materials  were  sent.  All  outbound  shipments 
were  recorded  in  the  NCS  Outbound  Mail  Management  system.  This  was  accomplished  by  having  a bar 
code  containing  the  school  number  on  each  address  label.  This  bar  code  was  read  into  the  system,  which 
determined  the  routing  of  the  shipment  and  the  charges.  Information  was  recorded  in  a file  on  the  system 
which,  at  the  end  of  each  day,  was  transferred  by  a PC  upload  to  the  mainframe.  A computer  program 
could  then  access  information  to  produce  reports  on  all  shipments  sent,  regardless  of  the  carrier  used. 
These  reports  helped  NCS  phone  staff  trace  shipments  for  Westat  supervisors  and  assessment 
administrators. 

A toll-free  telephone  line  was  maintained  for  supervisors  and  school  administrators  to  request 
additional  materials  for  the  National  assessments.  To  process  a shipment,  NCS  phone  staff  asked  the 
caller  for  information  such  as  primary  sampling  unit  (PSU),  school  ID,  assessment  type,  city,  state,  and 
ZIP  code.  This  information  was  then  entered  into  the  on-line  short  shipment  system  and  the  school’s 
mailing  address  would  be  displayed  on  the  screen  to  verify  with  the  caller.  The  system  allowed  NCS  staff 
to  change  the  shipping  address  for  individual  requests.  The  clerk  proceeded  to  the  next  screen  that 
displayed  the  materials  to  be  selected.  After  the  requested  items,  due  date  and  method  of  shipment  were 
entered,  the  system  produced  a packing  list  and  mailing  labels.  Phone  staff  also  took  phone  calls 
concerning  initial  shipment  delivery  dates,  tracing  a shipment,  and  questions  concerning  NAEP. 
Approximately  750  calls  were  received  regarding  the  1996  NAEP  assessments. 

Further  information  regarding  packaging  and  shipping  is  provided  in  the  Report  of  Processing 
and  Professional  Scoring  Activities  (National  Computer  Systems,  1996). 


6.4  PROCESSING 
6.4.1  Overview 


The  following  describes  the  various  stages  of  work  involved  in  receiving  and  processing  the 
documents  used  in  the  1996  assessment.  NCS  staff  created  a set  of  predetermined  rules  and  specifications 
for  the  processing  departments  within  NCS  to  follow.  Project  staff  performed  a variety  of  procedures  on 
materials  received  from  the  assessment  administrators  before  releasing  these  materials  into  the  NCS 
NAEP  processing  system.  Control  systems  were  used  to  monitor  all  NAEP  materials  returned  from  the 
field.  The  NAEP  Process  Control  System  (PCS)  contained  the  status  of  sampled  schools  for  all  sessions 
and  their  scheduled  assessment  dates.  As  materials  were  returned,  the  PCS  was  updated  to  indicate  receipt 
dates,  to  record  counts  of  materials  returned,  and  to  document  any  problems  discovered  in  the  shipments. 
As  documents  were  processed,  the  system  was  updated  to  reflect  processed  counts.  NCS  report  programs 
were  utilized  to  allow  ETS,  Westat,  and  NCS  staff  to  monitor  the  progress  in  the  receipt  control 
operations. 

An  “alert”  process  was  used  to  record,  monitor,  and  categorize  all  discrepant  or  problematic 
situations.  Throughout  the  processing  cycle,  alert  situations  were  either  flagged  by  computer  programs  or 
identified  during  clerical  check-in  procedures.  Certain  alerts,  such  as  missing  demographic  information 
on  the  administration  schedule,  were  resolved  by  opening  staff  retrieving  the  information  from  booklet 
covers.  These  alerts,  known  as  “Information  Alerts,”  were  recorded  directly  into  the  PCS  system  by 
opening  personnel,  eliminating  the  need  for  paper  documentation.  Since  these  problem  situations  were 
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categorized  and  tallied  as  they  were  key-entered  into  the  PCS  system,  project  staff  were  able  to  provide 
timely  reporting  on  clerical-type  errors  made  during  test  administration.  Alert  situations  that  could  not  be 
resolved  by  opening  personnel  were  described  on  alert  forms  that  were  forwarded  to  project  personnel  for 
resolution.  Once  resolved,  the  problems  and  resolutions  were  recorded  online  in  the  PCS  system. 

NCS’s  Work  Flow  Management  System  was  used  to  track  batches  of  student  booklets  through 
each  processing  step,  allowing  project  staff  to  monitor  the  status  of  all  work  in  progress.  It  was  also  used 
by  NCS  to  analyze  the  current  work  load,  by  project,  across  all  work  stations.  By  routinely  monitoring 
these  data,  NCS’s  management  staff  was  able  to  assign  priorities  to  various  components  of  the  work  and 
to  monitor  all  phases  of  the  data  receipt  and  processing. 


6.4.2  Document  Receipt 

Shipments  were  to  be  returned  to  NCS  packaged  in  their  original  boxes.  As  mentioned  earlier, 
NCS  packaging  staff  applied  a bar  code  label  to  each  box  indicating  the  NAEP  school  ID  number.  When  a 
shipment  arrived  at  the  NCS  dock  area,  this  bar  code  was  scanned  into  a personal  computer  file,  and  the 
shipment  was  forwarded  to  the  receiving  area.  The  file  was  then  transferred  to  the  mainframe  and  the 
shipment  receipt  date  was  applied  to  the  appropriate  school  within  the  PCS  system,  providing  the  status 
of  receipts  regardless  of  any  processing  delays.  Each  receipt  was  reflected  on  the  PCS  status  report 
provided  to  the  NCS  receiving  department  and  supplied  to  Westat  via  electronic  file  transfer  and  in  hard- 
copy format.  ETS  also  received  a hard  copy. 

The  PCS  file  could  be  manually  updated  to  reflect  changes.  Receiving  personnel  also  checked  the 
shipment  to  verify  that  the  contents  of  the  box  matched  the  school  and  session  indicated  on  the  label. 

Each  shipment  was  checked  for  completeness  and  accuracy.  Any  shipment  not  received  within  two  days 
of  the  scheduled  assessment  date  was  flagged  in  the  PCS  system  and  annotated  on  the  PCS  report.  The 
administration  status  of  these  delayed  shipments  was  checked  and  in  some  cases  a trace  was  initiated  on 
the  shipment. 

A new  requirement  for  NCS  was  to  open  all  shipments  within  forty-eight  hours  of  their  receipt 
and  to  key-enter  preliminary  processing  information  into  the  PCS  system  from  the  Administration 
Schedule.  The  preliminary  information  was  written  on  the  Administration  Schedule  by  Westat  assessment 
administrators  and  consisted  of  the  following: 

• School  number 

• Session  number 

• Original  test  date 

• Total  number  assessed 

This  preliminary  information,  used  to  provide  Westat  with  timely  student  response  rates,  was  updated 
with  actual  data  when  materials  passed  through  processing  error  free.  A completeness  flag  was  also 
applied  to  the  PCS  file  by  NCS  opening  staff  if  any  part  of  the  shipment  was  missing. 

If  multiple  sessions  were  returned  in  one  box,  the  contents  of  the  package  were  separated  by 
session.  The  shipment  was  checked  to  verify  that  all  booklets  preprinted  or  handwritten  on  the 
Administration  Schedule  were  returned  with  the  shipment  and  that  all  administration  codes  matched  from 
booklet  cover  to  the  Administration  Schedule.  If  discrepancies  were  discovered  at  any  step  in  this 
process,  the  receiving  staff  issued  an  alert  to  facilitate  tracking.  If  the  administrator  indicated  that  a make- 
up session  was  being  held  the  documents  were  placed  on  holding  carts  until  the  make-up  session 
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documents  arrived.  If  no  make-up  session  was  indicated,  Westat  was  contacted  for  the  status  of  the 
missing  materials.  If  the  missing  materials  were  to  be  returned,  the  documents  already  received  were  held 
until  that  time.  If  the  materials  were  not  being  returned,  processing  continued  and  the  appropriate 
administration  code  was  applied  to  the  Administration  Schedule. 


6.4.3  Batching  and  Scanning  Documents 

Once  all  booklets  listed  on  the  Administration  Schedule  for  a session  were  verified  as  present,  the 
entire  session  (both  the  Administration  Schedule  and  booklets)  was  batched  by  grade  level  and  session 
type.  Each  batch  was  assigned  a unique  batch  number.  This  number,  created  on  the  Image  Capture 
Environment  system  for  all  image-scannable  documents  and  on  the  Work  Flow  Management  system  for 
all  key-entry  and  OMR-scannable  documents^,  facilitated  the  internal  tracking  of  the  batches  and  allowed 
departmental  resource  planning.  All  other  scannable  documents  (School  Characteristics  and  Policies 
Questionnaires,  Teacher  Questionnaires,  Students  with  Disabilities/Limited  English  Proficiency 
[SD/LEP]  Questionnaires,  and  rosters)  were  batched  by  document  type  in  the  same  manner. 

Because  all  assessment  booklets  were  image-scannable,  batch  numbers  for  these  documents  were 
created  on  the  Image  Capture  Environment  system.  Sessions  were  sorted  by  grade  level  and  automatically 
uploaded  to  the  Work  Flow  Management  system  after  batch  creation.  The  Administration  Schedule  for 
these  document  types  was  used  as  a session  header  within  a batch. 

When  batching  mathematics  documents,  NCS  needed  to  allow  for  having  both  image-scannable 
and  key-entry  documents  present  in  the  same  session,  or  having  booklets  listed  on  the  Administration 
Schedule  that  would  not  be  present  in  processing.  This  was  due  to  the  testing  accommodations  of  large- 
print  and  Braille  that  were  key-entry  documents. 

Large-print  booklets  had  to  be  processed  separately  from  the  Administration  Schedule  and 
scannable  booklets  in  their  session.  A key-entry  session  header  was  created  for  these  booklets  that 
contained  the  school  ID  number  and  session  code  from  the  Administration  Schedule.  Long-term  trend 
reading/writing  booklets  were  processed  through  key-entry  with  the  same  type  of  key-entry  session 
header.  The  Administration  Schedules  from  reading/writing  sessions  were  processed  in  an 
Administration-Schedule-only  batch  through  the  image  scanning  system.  After  the  session  that  a large- 
print  booklet  came  with  passed  through  image  processing,  session  information  was  rejoined  within  the 
processing  computer  programs.  The  same  computerized  match  occurred  with  Trend  reading/writing 
materials  once  the  Administration-Schedule-Only  batch  that  contained  a session’s  administration 
schedule  passed  through  processing. 


6.4.4  Questionnaires 

The  long-term  trend  assessments  used  one  roster  to  account  for  all  questionnaires.  The  roster  of 
questionnaires  recorded  the  distribution  and  return  of  the  School  Characteristics  and  Policies 
Questionnaires  and  the  Excluded  Student  Questionnaires. 

The  main  assessments  utilized  one  roster  to  document  and  track  the  School  Characteristics  and 
Policies  Questionnaires  and  the  Students  with  Disabilities/Limited  English  Proficiency  (SD/LEP) 
Questionnaire.  In  addition,  the  main  and  state  assessments  used  the  roster  of  Teacher  Questionnaires  to 
record  the  distribution  and  return  of  Teacher  Questionnaires. 


^ OMR  is  the  acronym  for  Optical  Mark  Reading. 
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Some  questionnaires  may  not  have  been  available  for  return  with  the  shipment.  These  were 
returned  to  NCS  at  a later  date  in  an  envelope  provided  for  that  purpose.  The  questionnaires  were 
submitted  for  scanning  as  sufficient  quantities  became  available  for  batching. 

Receipt  of  the  questionnaires  was  entered  into  the  system  using  the  same  process  as  was  used  for 
the  Administration  Schedules  described  in  previous  sections.  The  rosters  were  grouped  with  other  rosters 
of  the  same  type  from  other  sessions,  and  a batch  was  created  on  the  Image  Capture  Environment  system. 
The  batch  was  then  forwarded  to  scanning  where  all  information  on  the  rosters  was  scanned  into  the 
system. 


6.4.5  Booklet  Accountability 

NCS  used  a sophisticated  booklet  accountability  system  to  track  all  distributed  booklets.  Prior  to 
the  distribution  of  NAEP  materials,  unique  booklet  numbers  were  read  by  bundle  into  a file.  Specific 
bundles  were  then  assigned  to  particular  supervisors  or  schools.  This  assignment  was  recorded  in  the 
NAEP  Materials  Distribution  System. 

When  shipments  arrived  at  NCS  from  the  field,  all  used  booklets  were  submitted  for  processing 
and  a “processed  documents”  file  was  maintained.  Unused  booklets  were  submitted  for  security  scanning 
where  booklet  ID  bar  codes  were  read  and  recorded  into  a separate  file.  This  file  and  the  “processed 
documents”  file  were  later  compared  to  the  original  bundle  security  file  for  individual  booklet  matching. 
A list  of  unmatched  booklet  IDs  was  printed  in  a report  used  to  confirm  non-receipt  of  individual 
booklets.  At  the  end  of  the  assessment  period,  supervisors  returned  all  unused  materials.  These  booklet 
IDs  were  also  read  by  the  bar-code  scanner  and  added  to  the  bundle  security  file.  All  unused  materials 
received  were  then  inventoried  and  sent  to  the  NCS  warehouse  for  storage  while  awaiting  authorization 
from  ETS  to  salvage  them. 


6.4.6  Data  Entry 

The  data  entry  process  was  the  first  point  at  which  booklet-level  data  were  directly  available  to 
the  computer  system.  Depending  on  the  NAEP  document,  one  of  three  methods  was  used  to  transcribe 
NAEP  data  to  a computerized  form.  The  data  on  scannable  documents  were  collected  using  NCS  optical- 
scanning  equipment  that  also  captured  images  of  the  constructed-response  items  and  ICR  fields. 
Nonscannable  materials  were  keyed  through  an  interactive  online  system.  In  both  of  these  cases,  the  data 
were  edited  and  suspect  cases  were  resolved  before  further  processing. 

All  student  booklets,  questionnaires,  and  control  documents  were  scannable.  Throughout  all 
phases  of  processing,  the  student  booklets  were  batched  by  grade  and  session  type.  The  scannable 
documents  were  then  transported  to  a slitting  area  where  the  folded  and  stapled  spine  was  removed  from 
the  document.  This  process  utilized  an  “intelligent  slitter”  to  prevent  slitting  the  wrong  side  of  the 
document.  The  documents  were  jogged  by  machine  so  that  the  registration  edges  of  the  NAEP  documents 
were  smoothly  aligned,  and  the  stacks  were  then  returned  to  the  cart  to  be  scanned. 

During  the  scanning  process,  each  scannable  NAEP  document  was  uniquely  identified  using  a 
print-after-scan  number  consisting  of  the  scan  batch  number,  the  sequential  number  within  the  batch,  and 
the  bar  code  ID  of  the  booklet.  These  numbers  were  printed  on  each  sheet  of  each  document  as  it  exited 
the  scanner.  This  permitted  the  data  editors  to  quickly  and  accurately  locate  specific  documents  during 
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the  editing  phase.  The  print-after-scan  number  remained  with  the  data  record,  providing  a method  for  easy 
identification  and  quick  retrieval  of  any  document. 

The  data  values  were  captured  from  the  booklet  covers  and  Administration  Schedules  and  were 
coded  as  numeric  data.  Unmarked  fields  were  coded  as  blanks  and  editing  staff  were  alerted  to  missing  or 
uncoded  critical  data.  Fields  that  had  multiple  marks  were  coded  as  asterisks  (*).  The  data  values  for  the 
item  responses  and  scores  were  returned  as  numeric  codes.  The  multiple-choice  single  response  format 
items  were  assigned  codes  depending  on  the  position  of  the  response  alternative;  that  is,  the  first  choice 
was  assigned  the  code  “1,”  the  second  “2,”  and  so  forth.  The  mark-all-that-apply  items  were  given  as 
many  data  fields  as  response  alternatives;  the  marked  choices  were  coded  as  “1”  while  the  unmarked 
choices  were  recorded  as  blanks.  The  images  of  constructed-response  items  were  saved  as  a digitized 
computer  file.  The  area  of  the  page  that  needed  to  be  clipped  was  defined  prior  to  scanning  through  the 
document  definition  process.  The  fields  from  unreadable  pages  were  coded  “X”  as  a flag  for  resolution 
staff  to  correct.  In  addition  to  capturing  the  student  responses,  the  bar  code  identification  numbers  used  to 
maintain  process  control  were  decoded  and  transcribed  to  the  NAEP  computerized  data  file. 

As  the  scanning  program  completed  scanning  each  stack,  the  stack  was  removed  from  the  output 
hopper  and  placed  in  the  same  order  they  were  scanned  on  the  output  cart.  The  next  stack  was  removed 
from  the  input  cart  and  placed  into  the  input  hopper,  after  which  the  scanning  resumed.  When  the 
operator  had  completed  processing  the  last  stack  of  the  batch,  the  program  was  terminated.  This  closed 
the  dataset  that  automatically  became  available  for  the  data  validation  (edit)  process.  The  scanned 
documents  were  then  forwarded  to  a holding  area  in  case  they  needed  to  be  retrieved  for  resolution  of  edit 
errors. 


NCS  again  used  the  ICR  engine  to  read  various  hand  and  machine  printing  on  the  front  cover  of 
the  assessment  and  supervisor  documents  for  the  1996  NAEP  assessments.  Some  information  from 
scannable  student  documents,  such  as  the  Administration  Schedule,  the  roster  of  questionnaires,  and  some 
questions  in  the  School  Characteristics  and  Policies  Questionnaires,  were  read  by  the  ICR  engine  and 
verified  by  an  online  key-entry  operator.  In  all,  the  ICR  engine  read  approximately  15  million  characters. 
The  ICR  engine  saved  NAEP  field  staff  and  school  personnel  a significant  amount  of  time  because  they 
no  longer  had  to  enter  these  data  by  gridding  rows  and  columns  of  data. 

NCS  also  implemented  new  programs  that  allowed  the  scanners  to  read  imprinted  codes,  known 
as  2-out-of-5  codes,  that  were  printed  via  a Xerox  4280  printer  on  the  Administration  Schedule.  These 
2-out-of-5  codes  were  imprinted  at  the  same  time  the  booklet  ID  numbers  were  printed  on  the 
Administration  Schedule  and  identified  which  booklet  IDs  were  listed  on  that  document.  When  the 
scanning  programs  were  unable  to  translate  the  2-out-of-5  codes  (thereby  identifying  the  booklet  ID 
numbers  on  the  document)  image  clips  of  the  booklet  ID  numbers  were  displayed  to  online  editing  staff 
for  verification.  This  eliminated  a significant  amount  of  online  editing  time  needed  to  process  the  NAEP 
assessments. 

To  provide  another  quality  check  on  the  image  scanning  and  scoring  system,  NCS  staff  stamped 
blank  booklets  with  a rubber  stamp  and  assigned  these  booklets  mock  scores  from  the  valid  range.  Each 
unique  item  type  scored  via  the  image  system  had  two  quality  control  stamps  per  valid  score.  An  example 
of  the  stamp  used  is  given  below. 
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The  quality  control  booklets  were  batched  and  processed  together  with  student  documents  of  the 
same  type.  Because  all  of  a specific  item  were  batched  together  for  transmission  to  the  scoring  facility, 
the  quality  control-stamped  responses  were  integrated  with  the  student  responses  and  transmitted 
simultaneously  to  the  scoring  facility.  During  the  scoring  process,  both  student  responses  and  the  quality 
control  items  were  randomly  displayed  so  scores  could  be  applied. 

When  a person  who  was  scoring  responses  (reader)  later  saw  the  quality  control  sample  on  the 
monitor  during  scoring,  he  or  she  was  to  notify  the  team  leader,  who  confirmed  the  score  assigned  by  the 
reader  was  the  score  listed  on  the  sample.  The  quality  control  booklets  were  included  in  the  pool  of  all 
items  to  be  drawn  from  for  the  25  percent  reliability  rescore. 

All  image  quality-assurance  documents  were  created  prior  to  the  beginning  of  scoring  and  all  pre- 
determined score  points  were  used.  Because  during  the  process  of  scoring,  valid  score  points  can  be 
changed  or  dropped  completely,  NCS  provided  ETS  with  documentation  explaining  what  quality  control 
documents  were  produced  and  which  score  points  on  these  items  were  no  longer  valid.  When  an  image 
quality  control  stamp  was  displayed  to  a reader  that  contained  a score  point  that  was  no  longer  valid,  the 
reader  gave  the  response  a score  point  of  zero. 

A process  of  key  entry  and  verification  was  used  to  make  corrections  to  the  non-scannable 
long-term  trend  reading/writing  documents  and  large  print  booklets.  Teacher  questionnaire  and  SD/LEP 
questionnaire  information  was  also  corrected  using  key-entry  methods.  NCS  used  the  Falcon  system  to 
enter  this  data.  The  Falcon  system  is  an  on-line  data-entry  system  designed  to  replace  most  methods  of 
data  input  such  as  keypunch,  key-to-disk,  and  many  of  the  microcomputer  data-entry  systems.  The 
terminal  screens  were  designed  to  enhance  operator  speed  and  convenience.  The  fields  to  be  entered  were 
titled  to  reflect  the  actual  source  document.  Therefore,  all  key-entry  fields  were  specific  to  the  NAEP 
student  documents  or  questionnaire  types  being  keyed. 

6.4.7  Data  Validation 

Each  dataset  produced  by  the  scanning  system  contained  data  for  a particular  batch.  These  data 
had  to  be  validated  (or  edited)  for  type  and  range  of  response.  The  data-entry  and  resolution  system  used 
was  able  to  simultaneously  process  a variety  of  materials  from  all  age  groups,  subject  areas,  control 
documents,  and  questionnaires  as  the  materials  were  submitted  to  the  system  from  scannable  and  non- 
scannable  media. 

The  data  records  in  the  scan  file  were  organized  in  the  same  order  in  which  the  paper  materials 
were  processed  by  the  scanner.  A record  for  each  batch  header  preceded  all  data  records  for  that  batch. 
The  document  code  field  on  each  record  distinguished  the  header  record  from  the  data  records. 

When  a batch-header  record  was  read,  a pre-edit  data  file  and  an  edit  log  were  generated.  As  the 
program  processed  each  record  within  a batch  from  the  scan  file,  it  wrote  the  edited  and  reformatted  data 
records  to  the  pre-edit  file  and  recorded  all  errors  on  the  edit  log.  The  data  fields  on  an  edit  log  record 
identified  each  data  problem  by  the  batch  sequence  number,  booklet  serial  number,  section  or  block  code, 
field  name  or  item  number,  and  data  value.  After  each  batch  had  been  processed,  the  program  generated  a 
listing  or  online  edit  file  of  the  data  problems  and  resolution  guidelines.  An  edit  log  listing  was  printed  at 
the  termination  of  the  program  for  all  non-image  documents.  Image  “clips”  requiring  editing  were  routed 
to  online  editing  stations  for  those  documents  that  were  image  scanned. 

As  the  program  processed  each  data  record,  it  first  read  the  booklet  number  and  checked  it  against 
the  session  code  for  appropriate  session  type.  Any  mismatch  was  recorded  on  the  error  log  and  processing 
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continued.  The  booklet  number  was  then  compared  against  the  first  three  digits  of  the  student 
identification  number.  If  they  did  not  match,  a message  was  written  on  the  error  log.  The  remaining 
booklet  cover  fields  were  read  and  validated  for  the  correct  range  of  values.  The  school  codes  had  to  be 
identical  to  those  on  the  PCS  record.  All  data  values  that  were  out  of  range  were  read  “as  is”  but  were 
flagged  as  suspect.  All  data  fields  that  were  read  as  asterisks  (*)  were  recorded  on  the  edit  log  or  online 
edit  file.  - 

Document  definition  files  described  each  document  as  a series  of  blocks  that  in  turn  were 
described  as  a series  of  items.  The  blocks  in  a document  were  transcribed  in  the  order  that  they  appeared 
in  the  document.  Each  block’s  fields  were  validated  during  this  process.  If  a document  contained  suspect 
fields,  the  cover  information  was  recorded  on  the  edit  log  along  with  a description  of  the  suspect  data. 

The  edited  booklet  cover  was  transferred  to  an  output  buffer  area  within  the  program.  As  the  program 
processed  each  block  of  data  from  the  dataset  record,  it  appended  the  edited  data  fields  to  the  data  already 
in  this  buffer. 

The  program  then  cycled  through  the  data  area  corresponding  to  the  item  blocks.  The  task  of 
translating,  validating,  and  reporting  errors  for  each  data  field  in  each  block  was  performed  by  a routine 
that  required  only  the  block  identification  code  and  the  string  of  input  data.  This  routine  had  access  to  a 
block  definition  file  that  had,  for  each  block,  the  number  of  fields  to  be  processed,  and,  for  each  field,  the 
field  type  (alphabetic  or  numeric),  the  field  width  in  the  data  record,  and  the  valid  range  of  values.  The 
routine  then  processed  each  field  in  sequence  order,  performing  the  necessary  translation,  validation,  and 
reporting  tasks. 

The  first  of  these  tasks  checked  for  the  presence  of  blanks  or  asterisks  (*)  in  a critical  field.  These 
were  recorded  on  the  edit  log  or  online  edit  file  and  processing  continued  with  the  next  field.  No  action 
was  taken  on  blank  fields  for  multiple-choice  items  because  the  asterisk  code  indicated  a non-response. 
The  field  was  validated  for  range  of  response,  and  any  values  outside  of  the  specified  range  were  recorded 
on  the  edit  log  or  online  edit  file.  The  program  used  the  item-type  code  to  make  a further  distinction 
among  constructed-response  item  scores  and  other  numeric  data  fields. 

Moving  the  translated  and  edited  data  field  into  the  output  buffer  was  the  last  task  performed  in 
this  phase  of  processing.  When  the  entire  document  was  processed,  the  completed  string  of  data  was 
written  to  the  data  file.  When  the  program  encountered  the  end  of  a file,  it  closed  the  dataset  and 
generated  an  edit  listing  for  non-image  and  key-entered  documents.  Image-scanned  items  that  required 
correction  were  displayed  at  an  online  editing  terminal. 

6.4.8  E)diting  for  Non-Image  and  Key-Entered  Documents 

Throughout  the  system,  quality  procedures  and  software  ensured  that  the  NAEP  data  were 
correct.  All  student  documents  on  the  Administration  Schedule  were  accounted  for,  as  receipt  control 
personnel  checked  that  the  materials  were  undamaged  and  assembled  correctly.  The  machine  edits 
performed  during  data  capture  verified  that  each  sheet  of  each  document  was  present  and  that  each  field 
had  an  appropriate  value.  All  batches  entered  into  the  system,  whether  key-entered  or  machine-scanned, 
were  edited  for  errors. 

Data  editing  took  place  after  these  checks.  This  consisted  of  a computerized  edit  review  of  each 
respondent’s  document  and  the  clerical  edits  necessary  to  make  corrections  based  upon  the  computer  edit. 
This  data-editing  step  was  repeated  until  all  data  were  correct. 
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The  first  phase  of  data  editing  was  designed  to  validate  the  population  and  ensure  that  all 
documents  were  present.  A computerized  edit  list,  produced  after  NAEP  documents  were  scanned  or  key 
entered,  and  all  the  supporting  documentation  sent  from  the  field  were  used  to  perform  the  edit  function. 
The  hard-copy  edit  list  contained  all  the  vital  statistics  about  the  batch:  number  of  students,  school  code, 
type  of  document,  assessment  code,  suspect  cases,  and  record  serial  numbers.  Using  these  inputs,  the  data 
editor  verified  that  the  batch  had  been  assembled  correctly  and  that  each  school  number  was  correct. 

During  data  entry,  counts  of  processed  documents  were  generated  by  type.  These  counts  were 
compared  against  the  information  captured  from  the  Administration  Schedules.  The  number  of  assessed 
and  absent  students  processed  had  to  match  the  numbers  indicated  on  the  PCS. 

In  the  second  phase  of  data  editing,  experienced  editing  staff  used  a predetermined  set  of 
specifications  to  review  the  field  errors  and  record  necessary  corrections  to  the  student  data  file.  The 
same  computerized  edit  list  used  in  phase  one  was  used  to  perform  this  function.  The  editing  staff 
reviewed  the  computer-generated  edit  log  and  the  area  of  the  source  document  that  was  noted  as  being 
suspect  or  as  containing  possible  errors.  The  composition  of  the  field  was  shown  in  the  edit  box.  The 
editing  staff  checked  this  piece  of  information  against  the  NAEP  source  document.  At  that  point,  one  of 
the  following  took  place: 

Correctable  error.  If  the  error  was  correctable  by  the  editing  staff  as  per  the  editing 
specifications,  the  correction  was  noted  on  the  edit  log  for  later  correction  via  key-entry. 

Alert.  If  an  error  was  not  correctable  as  per  the  specifications,  an  alert  was  issued  to 
NAEP  project  staff  for  resolution.  Once  the  correct  information  was  obtained,  the 
correction  was  noted  on  the  edit  log  for  key-entry  correction. 

Non-correctable  error.  If  a suspected  error  was  found  to  be  correct  as  stated  and  no 
alteration  was  possible  according  to  the  source  document  and  specifications,  the 
programs  were  tailored  to  allow  this  information  to  be  accepted  into  the  data  record.  No 
corrective  action  was  taken. 

The  corrected  edit  log  was  then  forwarded  to  the  key-entry  staff  for  processing.  When  all 
corrections  were  entered  and  verified  for  a batch,  an  extract  program  pulled  the  corrected  records  into  a 
mainframe  dataset.  At  this  point,  the  mainframe  edit  program  was  initiated.  The  edit  criteria  were  again 
applied  to  all  records.  If  there  were  further  errors,  a new  edit  listing  was  printed  and  the  cycle  was 
repeated. 

When  the  edit  process  produced  an  error-free  file,  the  booklet  ID  number  was  posted  to  the 
NAEP  tracking  file  by  age,  assessment,  and  school.  This  permitted  NCS  staff  to  monitor  the  NAEP 
processing  effort  by  accurately  measuring  the  number  of  documents  processed  by  form.  The  posting  of 
booklet  IDs  also  ensured  that  a booklet  ID  was  not  processed  more  than  once. 


6.4.9  Data  Validation  and  Editing  of  Image-Processed  Documents 

The  paper  edit  log  for  key-entered  documents  was  replaced  by  online  viewing  of  suspect  data  for 
all  image-processed  documents.  For  rapid  resolution,  the  edit  criteria  for  each  item  in  question  appeared 
on  the  screen  along  with  the  suspect  item.  Corrections  were  made  immediately.  The  system  employed  an 
edit/verify  system  that  ultimately  meant  that  two  different  people  viewed  the  same  suspect  data  and 
operated  on  it  separately.  The  “verifier”  made  sure  the  two  responses  (one  from  either  the  entry  operator 
or  the  ICR  engine)  were  the  same  before  the  system  accepted  that  iteifi  S^^ing  correct.  The  verifiers 


could  either  overrule  or  agree  with  the  original  correction  made  if  the  two  did  not  match.  If  the  editor 
could  not  determine  the  appropriate  response,  he  or  she  escalated  the  suspect  situation  to  a supervisor.  For 
errors  or  suspect  information  that  could  not  be  resolved  by  supervisory  staff,  a product-line  queue  was 
created.  This  allowed  supervisors  to  escalate  edits  to  project  staff  for  resolution.  By  having  this  product- 
line queue,  project  staff  were  able  to  quickly  locate  edit  clips  within  the  image  system,  speeding  up  the 
resolution  process. 

Once  an  entire  batch  was  through  the  edit  phase,  it  became  eligible  for  the  count-verification 
phase.  The  Administration  Schedule  data  were  examined  systematically  for  booklet  IDs  that  should  have 
been  processed  (assessed  administration  codes).  All  documents  under  that  Administration  Schedule  were 
then  inspected  to  ensure  that  all  of  the  booklets  were  included. 

With  the  satisfactory  conclusion  of  the  count-verification  phase,  the  edited  batch  file  was 
uploaded  to  the  mainframe,  where  it  went  through  yet  another  edit  process.  A paper  edit  log  was  produced 
and,  if  errors  remained,  was  forwarded  to  another  editor.  When  this  paper  edit  was  satisfied,  the  PCS  and 
Workflow  Management  system  were  updated.  Because  there  was  a possible  time  lag  between  a clean  edit 
in  the  image  system  and  a clean  edit  in  the  mainframe  systems,  the  batch  was  not  archived  until  48  hours 
after  the  image  edit  phase  was  completed. 


6.4.10  Data  Transmission 

Due  to  the  rapid  pace  of  scoring  on  an  item-by-item  basis,  the  NCS  scoring  specialists  found  it 
necessary  to  continually  monitor  the  status  of  work  available  to  the  readers  and  plan  the  scoring  schedule 
several  weeks  in  advance.  On  Wednesday  of  each  week,  the  NGS  performance  assessment  specialist  in 
charge  of  each  subject  area  planned  the  next  two  weeks’  schedule.  That  information  was  then  provided  to 
the  person  in  charge  of  downloading  data  to  the  scoring  center.  By  planning  the  scoring  schedule  two 
weeks  in  advance,  the  scoring  specialists  were  able  to  ensure  that  readers  would  have  sufficient  work  for 
at  least  one  week,  after  which  the  next  download  would  occur  to  supplement  the  volume  of  any  unscored 
items  and  add  an  additional  week’s  work  to  the  pool  of  items  to  score.  Additionally,  by  scheduling  two 
weeks’  data  transmission,  flexibility  was  added  to  the  scoring  schedule,  making  it  possible  to  implement 
last-minute  changes  in  the  schedule  once  the  items  had  been  delivered  to  the  scoring  center.  Depending 
on  the  number  of  items  to  be  transmitted,  the  actual  downloading  was  conducted  on  Friday  or  was  divided 
into  two  smaller  sessions  for  Thursday  and  Friday  download.  By  the  first  week  of  May  1996,  there  was 
sufficient  space  on  the  scoring  servers  to  load  all  remaining  unscored  items  to  the  scoring  center. 

Delivery  of  data  to  the  scoring  center  was  accomplished  via  several  T1  transmission  lines  linking 
the  mainframe  computers  and  the  NAEP  servers  at  the  document-scanning  site  in  the  NCS  main  facility 
with  the  scoring  servers  dedicated  to  distributing  work  to  the  professional  readers  at  the  scoring  center. 
The  actual  task  of  scheduling  items  for  downloading  was  accomplished  using  a code  written  by  the  Image 
Software  Development  team.  This  code  enabled  the  person  scheduling  the  download  to  choose  a team  of 
readers  and  select  the  scheduled  items  from  a list  of  all  items  that  that  team  would  be  scoring  throughout 
the  scoring  project.  This  process  was  repeated  for  all  teams  of  readers  until  all  anticipated  work  was 
scheduled.  Once  this  task  was  completed,  the  scheduled  job  was  tested  to  determine  if  there  was 
sufficient  free  disk  space  on  the  servers  at  the  scoring  center.  If  for  any  reason  sufficient  disk  space  was 
not  available,  scheduled  items  could  be  deleted  from  the  batch  individually  or  as  a group  until  the 
scheduled  batch  job  could  accommodate  all  items  on  the  available  disk  space  at  the  scoring  center.  Once 
it  was  determined  that  sufficient  disk  space  was  available,  transmission  of  student  responses  commenced. 
Data  transmission  was  typically  accomplished  during  off-shift  hours  to  minimize  the  impact  on  system- 
load capacity. 
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6.5  DATA  DELIVERY 


The  1996  NAEP  data  collection  resulted  in  several  classes  of  data  files  — student,  school, 
teacher,  weights,  SD/LEP  student,  excluded  student  questionnaire  data  for  long-term  trends, 
student/teacher  match,  and  student-response  information.  Student-response  information  included  response 
data  from  all  assessed  students  in  1996.  Data  resolution  activities  occurred  prior  to  the  submission  of  data 
files  to  ETS  and  Westat  to  resolve  any  irregularities  that  existed.  This  section  details  additional  steps 
performed  before  creating  the  final  data  files  to  ensure  capture  of  the  most  complete  and  accurate 
information. 

An  important  quality-control  component  of  the  image-scoring  system  was  the  inclusion,  for 
purposes  of  file  identification,  of  an  exact  copy  of  the  student  edit  record,  including  the  student  booklet 
ID  number,  with  every  image  of  a student’s  response  to  a constructed-response  item.  These  edit  files  also 
remained  in  the  main  data  files  residing  on  the  NCS  mainframe  computer.  By  doing  this,  exact  matching 
of  scores  assigned  to  constructed-response  items  and  all  other  data  for  each  individual  student  was 
guaranteed,  since  the  booklet  ID  for  each  image  was  part  of  every  image  file.  This  ensured  scores  were 
applied  to  the  correct  student’s  record  on  the  mainframe. 

When  all  of  the  responses  for  an  individual  item  had  been  scored,  the  system  automatically 
submitted  all  item  scores  assigned  during  scoring,  along  with  their  edit  records,  to  a queue  to  be 
transmitted  to  the  mainframe.  Project  staff  then  initiated  a system  job  to  transmit  all  scoring  data  to  be 
matched  with  the  original  student  records  on  the  mainframe.  A custom  edit  program  matched  the  edit 
records  of  the  scoring  files  to  those  of  the  original  edit  records  on  the  mainframe.  As  matches  were 
confirmed,  the  scores  were  applied  to  those  individual  files.  After  completion  of  this  stage,  all  data 
collected  for  an  individual  student  was  located  in  one  single  and  complete  record/file  identified  by  the 
edit  record. 

NCS  processed  the  SD/LEP  student  questionnaires  via  OMR  scanning.  Edits  performed  on  the 
questionnaires  assured  that  responses  to  questions  fell  within  the  valid  range  for  that  question.  SD/LEP 
questionnaires  were  then  matched  to  a student  record.  SD/LEP  Questionnaires  that  were  not  matched  to  a 
student  document  were  cross-referenced  with  the  corresponding  Administration  Schedule,  roster  of 
questionnaires,  and  student  data  files  to  correct,  if  necessary,  the  information  needed  to  result  in  a match. 

In  1996,  NCS  continued  to  use  ICR  technology  to  capture  percentage  figures  written  by  school 
personnel  directly  in  boxes  on  the  school  characteristics  and  policies  questionnaires  rather  than  requiring 
the  school  official  to  grid  ovals  in  a matrix.  The  data  were  then  verified  by  an  edit  operator. 

To  achieve  the  best  possible  student/teacher  match  rate,  the  same  processes  that  were  followed  in 
previous  cycles  were  used  in  1996.  The  first  step  was  to  identify  teacher  questionnaires  not  returned  to 
NCS  for  processing  so  as  to  exclude  from  the  matching  process  the  students  of  these  teachers.  Student 
identification  numbers  that  were  not  rnatched  to  a teacher  questionnaire  were  cross-referenced  with  the 
corresponding  Adniinistration  Schedule  and  roster  of  teacher  questionnaires  to  verify  (and  change,  if 
necessary)  the  teacher  number,  teacher  period,  and  questionnaire  number  recorded  on  these  control 
documents.  The  NAEP  school  numbers  listed  on  the  roster  of  questionnaires  and  teacher  questionnaire 
were  verified  and  corrected,  if  necessary.  Once  these  changes  were  made,  any  duplicate  teacher  numbers 
existing  within  a school  were,  if  possible,  cross-referenced  for  resolution  with  the  roster  of 
questionnaires.  Since  this  information  was  located  together  on  a single,  central  control  document,  the 
ability  to  match  and  resolve  discrepant  or  missing  fields  was  simplified. 
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After  all  data-processing  activities  were  completed,  data  cartridges  and/or  diskettes  were  created 
and  shipped  via  overnight  delivery  to  ETS  and/or  Westat.  A duplicate  archive  file  is  maintained  at  NCS 
for  security  and  backup  purposes. 


6.6  MISCELLANEOUS 

6.6.1  Storage  of  Documents 

After  batches  of  image-scanned  documents  had  successfully  passed  the  editing  process,  they  were 
sent  to  the  NCS  warehouse  for  storage.  The  long-term  trend  reading/writing  booklets  were  sent  to  the 
NCS  scoring  center  to  be  scored  on  paper  and  to  be  used  for  holistic  and  mechanics  scoring.  After  all 
scoring  had  been  completed  for  the  long-term  trend  booklets,  they  were  also  sent  to  the  warehouse  for 
storage.  Due  to  the  large  number  of  rescore  projects  done  with  NAEP  material,  the  documents  were 
unspiraled  and  sequenced  by  grade  and  booklet  type  after  all  of  the  processing/scoring  was  completed. 
This  will  allow  for  efficient  document  retrieval  to  fill  requests  for  specific  booklet  types  for  future 
scoring  projects.  Unspiraled  and  sequenced  booklets  were  then  assigned  a new  inventory  number  by 
grade  and  booklet  type  and  sent  back  to  the  warehouse  for  storage.  The  storage  locations  of  all  documents 
were  recorded  on  the  inventory  control  system.  Unused  materials  were  sent  to  temporary  storage  to  await 
completion  of  the  entire  assessment.  Once  the  assessment  was  complete,  NCS  received  authorization 
from  ETS  to  salvage  unused  materials  after  determining  that  a sufficient  quantity  of  each  form  type  was 
retained  in  permanent  archive. 


6.6.2  Quality-Control  Documents 

ETS  requires  that  a random  sample  of  booklets  and  the  corresponding  scores/scoring  sheets  be 
pulled  for  an  additional  quality-control  check.  For  image-scanned  documents,  a scoring  sheet  is  not  used, 
so  ETS  uses  scores  sent  to  them  on  a data  tape  to  verify  the  accuracy  of  applied  scores.  During  the 
scoring  of  mathematics,  a selected  number  of  image-processed  booklets  were  paper  scored.  If  any  of  the 
random  sample  of  mathematics  booklets  used  for  paper  scoring  were  selected  as  quality-control 
documents,  the  scoring  sheet  was  also  sent  to  ETS.  For  non-scannable  trend  reading/writing  booklets  and 
for  the  trend  mathematics/ science  booklets  that  were  scored  via  paper,  both  the  booklet  and  its 
corresponding  score  sheet  were  sent.  All  of  these  documents  were  selected  prior  to  sending  the  booklets 
to  storage  and  were  then  sent  to  ETS  to  verify  the  accuracy  and  completeness  of  the  data.  A random 
sample  of  all  the  questionnaires  used  in  the  1996  NAEP  assessment  was  also  sent  to  ETS. 


6.6.3  Alert  Analysis 

Table  6-5  identifies  the  different  types  of  alerts  to  problems  that  were  encountered  in  the 
processing  of  NAEP  data.  For  the  1996  main  and  long-term  trend  assessments,  there  was  a total  of  230 
alerts;  for  the  State  Assessment,  there  was  a total  of  3,812  alerts. 

Discrepancies  were  found  in  the  receiving  process  that  did  not  require  an  alert  to  be  issued  to 
Westat.  They  did  require  a great  deal  of  effort  by  the  opening  staff  to  resolve  in  order  to  provide  the  most 
complete  and  accurate  information.  These  are  referred  to  as  “info  alerts.”  These  were  categorized  and 
codes  were  assigned  to  them.  They  are  listed  in  the  left-hand  column  of  Table  6-5. 
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Even  though  receipt-control  staff  were  well  trained  in  the  resolution  of  many  situations,  there 
were  some  problems  that  required  resolution  by  NCS  NAEP  product  line  staff.  These  are  referred  to  as 
“problem  alerts.”  The  various  types  of  problem  alerts  were  also  categorized  and  coded.  They  are  listed  in 
the  right-hand  column  of  Table  6-5.  For  any  unusual  situations,  Westat  was  contacted  to  help  with  the 
resolution  of  the  alert. 


Table  6-5 

Alerts  for  the  1996  National  and  State  Assessments 


Information  Alerts 

Problem  Alerts 

Code  52  not  written  on  Administration 

Change  of  Administration  Codes-A/S  or 

Schedules 

Booklets 

The  yes/no  box  not  gridded  on  Rosters 

Incorrect  Rosters/Questionnaires 

Session  Number  not  on  Administration  Schedules 

Administration  Notes/W riting  on  Covers 

Administration  Codes  not  on  A/S;  but  on  booklets 

Duplicate  Student  / Booklet  Number/ 
Administration  Schedule 

Administration  Codes  not  on  booklets;  but  on  A/S 

All  material  not  returned 

Items  returned  for  Westat 

Affected  Testing  - Problem 

Writing  on  booklet  covers 

Transcribed  page(s)  for  student  booklet(s) 

Other 

Processed  as  is 

Involves  Inclusion  Check  List 
Other 

A/S  = Administration  Schedules 
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Chapter  7 

PROFESSIONAL  SCORING^ 

Patrick  B.  Bourgeacq,  Bradley  Thayer,  and  Timothy  Robinson 
National  Computer  Systems 


7.1  INTRODUCTION 

The  1996  national  assessment  required  the  scoring  of  constructed  responses  in  mathematics  and 
science  at  grades  4,  8,  and  12.  Long-term  trend  assessments  for  9-,  13-,  and  17-year-olds  in  reading, 
writing,  mathematics  and  science  continued  at  levels  comparable  to  previous  assessment  years.  In  all, 
over  nine  million  constructed  responses  for  the  1996  national  and  state  assessments  werq  scored. 

More  than  300  professional  readers  split  between  a full-time  day  shift  and  part-time  evening  shift 
were  hired  for  scoring.  Veteran  scorers  were  mixed  with  new  hires  screened  for  their  ability  to  score 
constructed  responses,  providing  the  scoring  center  with  excellent  groups  of  qualified  readers.  Regular 
full-time  staff,  with  the  help  of  administrative  assistants,  bridged  the  shifts  to  ensure  quality  scoring 
between  the  two  groups.  For  the  first  time  in  a National  Assessment  of  Educational  Progress  (NAEP) 
assessment.  National  Computer  Systems  (NCS)  provided  a significant  number  of  trainers  who  worked 
with  Educational  Testing  Service  (ETS)  staff  to  train  teams  of  scorers  on  many  items.  Also,  NCS  used 
lead  scorers  for  the  first  time  in  a NAEP  assessment  to  assist  the  table  leaders  with  administrative  duties 
and  monitoring  quality  of  scoring.  The  help  of  lead  scorers  made  it  possible  to  score  greater  volumes  of 
responses  with  teams  as  large  as  14  scorers  without  any  apparent  compromises  in  the  quality  of  scoring. 

As  in  previous  years,  the  image  system  distributed  responses  among  teams  who  scored  them  in 
an  efficient  manner  to  maximize  consistency  and  reliability.  The  system  also  provided  enhanced  tools  to 
display  images,  gather  data,  and  monitor  the  quantity  and  quality  of  work. 

The  figures  and  tables  on  the  following  pages  summarize  the  scope  of  professional  scoring  for 
the  1996  NAEP. 


‘ Bradley  Thayer  was  the  NCS  project  manager  for  1996  NAEP,  Patrick  Bourgeacq.was  the  NCS  project  director  for  1996  NAEP 
scoring,  and  Timothy  Robinson  was  the  NCS  senior  processing  coordinator  for  1996  NAEP.  • 
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Figure  7-1 

Image  Scoring  Flow  Chart 
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Figure  7-2 

Paper  Scoring  Flow  Chart 
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7.2  LONG-TERM  TREND  ASSESSMENTS 
7.2.1  Mathematics 

Items  that  contributed  to  long-term  trend  in  mathematics  items  were  scored  as  “right,”  “wrong,” 
or  “omitted.”  The  scoring  criteria  identified  the  correct  or  acceptable  answers  for  each  item  in  each 
block.  The  scores  for  these  items  included  a “0”  for  no  response,  a “1”  for  a correct  answer,  or  a “2”  for 
an  incorrect  or  “I  don’t  know”  response.  The  reading  items  that  appeared  in  the  mathematics/science 
booklets  were  scored  as  “attempted,”  or  “omitted.”  This  scoring  consisted  of  merely  checking  to  see 
whether  the  student  had  responded  in  any  way  to  that  item,  in  which  case  the  item  was  determined  to 
have  been  reached  or  attempted.  The  scoring  here  was  “0”  for  not  attemipting  the  item  (blank)  or  “1”  for 
any  writing  in  the  space  provided.  This  includes  one  reading  item  in  an  age  9 math  booklet  and  one 

reading  item  in  an  age  13  math  booklet.  The  numbers  of  discrete  constructed-response  mathematics  items 
can  be  found  in  Table  7-1. 

Since  scoring  of  the  long-term  trend  mathematics  items  was  identical  to  previous  years,  no  new 
training  papers  were  needed.  Preparation  for  scoring  included  copying  the  scoring  guides  from  previous 
assessment  years  and  drawing  samples  from  previous  years,  retrieving  the  booklets  listed  in  the  samples, 
and  printing  and  matching  scoring  sheets  for  those  booklets. 

Because  the  mathematics  items  were  scored  as  “right,”  “wrong,”  or  “omitted,”  lengthy  training 
for  scoring  these  items  was  unnecessary.  For  each  component  (fall,  winter,  and  spring),  a different  team 
was  trained  to  follow  the  procedures  for  scoring  the  mathematics  items  and  became  familiar  with  the 
scoring  standards,  which  listed  general  guidelines  and  also  the  correct  answer  for  the  items  in  each  of  the 
blocks.  Each  season,  the  entire  scoring  was  done  in  one  or  two  weeks  at  the  end  of  the  administration 
period.  The  number  of  booklets  processed  and  the  number  of  constructed  responses  scored  for  each  age 
level  are  reported  in  Table  6-1  in  Chapter  6. 

A different  team  scored  each  age  level  at  the  time  of  year  the  age  level  was  assessed.  The 
booklets  arrived  in  sessions,  so  each  reader  scored  all  items  in  all  mathematics  booklet  types  during  the 
course  of  the  project.  All  scorers  held  the  same  qualifications  as  the  readers  for  main  and  state 
assessments.  The  number  of  readers,  table  leaders,  and  dates  of  scoring  are  reported  in  Table  7-2. 

To  establish  the  consistency  of  scoring  across  years,  the  readers  rescored  a subset  of  the 
responses  from  previous  assessments.  Samples  of  350  responses  to  each  item  from  the  1990  assessment 
and  250  from  the  1994  assessment  were  drawn.  The  Performance  Assessment  Scoring  Center  (PSC) 
score  sheet  scanning  system  gave  real-time  reports  comparing  the  original  scores  to  the  scores  assigned 
by  this  year’s  team.  The  team  also  second  scored  33  percent  of  the  current  year  sample  to  measure 
consistency  of  scoring.  The  table  leaders  monitored  daily  interreader  agreement  reports  and  t-tests  to 
verify  consistency  of  scores  within  year  and  across  years.  Summaries  of  the  interreader  agreement  figures 
can  be  found  in  Table  7-3. 
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Table  7-1 


Number  of  Constructed-Response  Items  by  Score-Point  Levels 


Subject 

2- 

3- 

4- 

5- 

6- 

Age/Grade  Category 

Category  Category  Category  Category 

Total 

Mathematics  Long-Term  Trend 

Age  9 

29 

0 

0 

0 

0 

29 

Age  13 

28 

0 

0 

0 

0 

28 

Age  17 

29 

0 

0 

0 

0 

29 

Total 

86 

0 

0 

0 

0 

86 

Reading  Long-Term  Trend 

. Age.  Class  9 

9 

0 

1 

2 , 

2 

14 

Age  Class  13 

8 

0 

2 

4 . 

1 

15 

Age  Class  17 

8 

0 

3 

4 

1 

16 

Total 

25 

0 

6 

10 

4 

45 

Writing  Long-Term  Trend 

Age  Class  9 

0 

2 

4 

0 

0 

6 

Age  Class  13 

1 

1 

5 

0 

0 

7 

Age  Class  17 

3 

0 

6 

0 

0 

9 

Total 

4 

3 

15 

0 

0 

22 

Writing  Long-Term  Trend  (Holistic  Scoring) 

. Age  Class  9 

0 

0 

0 

0 

2 

2 

Age  Class  13 

0 

0 

0 

0 

2 

2 

Age  Class  17 

0 

0 

0 

0 

2 

2 

Total 

0 

0 

0 

0 

6 

6 

Bilingual  Mathematics 

Grade  4 

1 

7 

1 

1 

0 

10 

Grade  8 

1 

9 

2 

0 

0 

12 

Total 

2 

16 

3 

1 

0 

22 

Mathematics  New  Items 

Grade  4 

0 

18 

14 

7 

0 

39 

Grade  4/8 

0 

5 

0 

0 

0 

5 

Grade  8 

5 

30 

5 

6 

0 

46 

Grade  8/12 

0 

4 

1 

2 

0 

7 

Grade  12 

2 

29 

4 

8 

0 

43 

Grade  4/8/12 

0 

0 

0 

0 

0 

0 

Total 

7 

86 

24 

23 

0 

140 

Mathematics  Short-Term  Trend  (Base  Year  1990) 

Grade  4 

0 

18 

14 

7 

0 

39 

Grade  4/8 

0 

5 

0 

0 

0 

5 

Grade  8 

5 

30 

5 

6 

0 

46 

Grade  8/12 

0 

4 

1 

2 

0 

7 

Grade  12 

2 

29 

4 

8 

0 

43 

Grade  4/8/12 

0 

0 

0 

0 

0 

0 

Total 

7 

86 

24 

. 23 

0 

140 

(continued) 
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Table  7-1  (continued) 


Number  of  Constructed-Response  Items  by  Score-Point  Levels 

Subject 

2- 

3- 

4- 

5- 

6- 

Age/Grade 

Category 

Category  Category  Category  Category 

Total 

Mathematics  Short-Term  Trend  (Base  Year  1992) 
Grade  4 54 

2 

2 

0 

13 

Grade  4/8 

1 

1 

2 

5 

0 

9 

Grade  8 

5 

4 

1 

3 

0 

13 

Grade  8/12 

1 

1 

0 

0 

0 

2 

Grade  12 

2 

3 

4 

4 

0 

13 

Grade  4/8/12 

0 

3 

1 

0 

0 

4 

Total 

14 

16 

10 

14 

0 

54 

Science 

Grade  4 

7 

52 

10 

2 

0 

71 

Grade  4/8 

2 

47 

8 

1 

0 

58 

Grade  8 

1 

41 

17 

0 

0 

59 

Grade  8/12 

2 

50 

8 

0 

0 

60 

Grade  12 

4 

83 

32 

6 

1 

126 

Total 

16 

273 

75 

9 

1 

374 

Table  7-2 

Professional  Scoring  - Readers  and  Dates 

Assessment 

Number  of 
Table  Leaders 

Number  of 
Scorers 

Dates 

Fall  Long-Term  Trend  Reading/Writing 

1 

7 

11/29/95-1/5/96 

Fall  Long-Term  Trend  Mathematics 

0 

11 

12/18/95-1/5/96 

Winter  Long-Term  Trend  Reading/Writing 

1 

6 

1/30/96-4/3/96 

Winter  Long-Term  Trend  Mathematics 

1 

9 

3/25/96-3/29/96 

Spring  Long-Term  Trend  Reading/Writing 

1 

6 

4/10/96-5/24/96 

Spring  Long-Term  Trend  Mathematics 

1 

5 

5/13/96-5/24/96 

Long-Term  Trend  Writing  Holistic 

■ 7 

56 

6/10/96-61/6/96 

Long-Term  Trend  Writing  Mechanics 

6 

33 

7/15/96-7/31/96 

Mathematics  Segment  1 Days 

8 

72 

3/13/96-4/5/96 

Mathematics  Segment  2 Days 

9 

126 

4/8/96-5/6/96 

Mathematics  Segment  2 Evenings 

5 

70 

4/8/96-5/2/96 

Bilingual  Mathematics 

0 

4 

5/2/96-5/3/96 

Science  Segment  1 Days 

3 

24 

3/18/96-4/5/96 

Science  Segment  2 Days 

3 

27 

4/8/96-5/3/96 

Science  Segment  2 Evenings 

5 

70 

4/8/96-5/2/96 

Science  Segment  3 Days 

12 

156 

5/6/96-6/7/96 

Science  Segment  3 Evenings 

12 

150 

5/6/96-6/7/96 

146 


169 


Table  7-3 

Interreader  Reliability  Ranges 


Assessment^ 

Number  of 
Unique  Items 
Total 

Number  of  Items  in 
Percentage  Exact  Agreement  Range 
60-69  % 70-79%  80-89%  Above  90% 

Fall  Long-Term  Trend  ReadingAVriting 

13 

0 

0 

3 

10 

Fall  Long-Term  Trend  Mathematics 

28 

0 

0 

0 

28 

Winter  Long-Term  Trend  ReadingAVriting 

11 

0 

0 

0 

11 

Winter  Long-Term  Trend  Mathematics 

29 

0 

0 

0 

29 

Spring  Long-Term  Trend  ReadingAVriting 

14 

0 

0 

6 

8 

Winter  Long-Term  Trend  Mathematics 

29 

0 

0 

0 

29 

Long-Term  Trend  Writing  Holistic^ 

6 

0 

0 

1 

5 

Long-Term  Trend  Writing  Mechanics 

3 

N/A 

N/A 

N/A 

N/A 

4th  Grade  Mathematics 

80 

0 

0 

2 

78 

8th  Grade  Mathematics 

102 

0 

0 

4 

98 

12th  Grade  Mathematics 

100 

0 

1 

9 

90 

4th  Grade  Science 

94 

0 

0 

13 

81 

8th  Grade  Science 

125 

0 

0 

20 

105 

12th  Grade  Science 

156 

0 

0 

26 

130 

’ Not  all  long-term  trend  items  received  second  scoring.  Figures  are  included  here  only  for  those  that  were  second  scored. 
^ Figures  for  long-term  trend  writing  holistic  include  adjacent  scores. 


7.2.2  Reading  and  Writing  (Primary  Trait) 

All  of  the  writing  items  for  the  three  long-term  trend  assessments  (fall,  winter,  and  spring)  were 
scored  using  the  primary  trait  method.  This  method  focused  on  the  writer  s effectiveness  in 
accomplishing  specific  assigned  tasks.  The  primary  trait  scoring  criteria  defined  five  levels  of  task 
accomplishment: 


1 . not  rated, 

2.  unsatisfactory, 

3.  minimal, 

4.  adequate,  and 

5.  elaborated. 

The  scoring  standard  for  each  item  described  these  levels  in  detail.  Some  of  these  items  were  also  scored 
for  secondary  traits,  which  involved  indicating  the  presence  or  absence  of  elements  that  were  of  special 
significance  to  a particular  item  (e.g.,  whether  notes  were  made  before  writing  or  whether  critical 
information  was  filled  out  on  a form). 

The  scoring  guides  for  the  constructed-response  reading  items  focused  on  students’  abilities  to 
perform  various  reading  tasks: 

• identifying  the  author’s  message  or  mood  and  substantiating  their  interpretation, 

• making  predictions  based  on  given  details,  and 

• comparing  and  contrasting. 
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The  guides  for  the  reading  items  varied  somewhat,  but  typically  included  a range  of  scores  denoting 
inability  to  address  the  task,  unsatisfactory  responses,  minimal  ability  in  accomplishing  the  task, 
satisfactory  ability  in  addressing  the  task,  or  elaborated  responses  addressing  the  task  fully.  Some  of  the 
reading  items  received  scores  for  secondary  traits  based  on  what  reactions  or  information  the  student 
gave  (i.e.,  whether  the  response  was  mostly  content  based,  form  based,  a subjective  reaction,  or  some 
combination  of  the  three). 

The  scoring  guides  for  the  constructed-response  writing  items  focused  on  students’  abilities  to 
write  in  informative,  persuasive,  and  narrative  styles.  The  guides  for  the  writing  items  were  based  on  a 
range  of  scores  denoting  unsatisfactory  writing  to  address  the  task,  minimal  writing  to  address  the  task, 
satisfactory  writing  to  address  the  task,  and  elaborated  writing  to  address  the  task. 

The  item  known  as  “The  Door”  was  scored  for  attemptedness  only.  The  readers  coded  all  blanks 
as  “0”  and  any  attempt  to  answer  as  a “1.” 

The  numbers  of  discrete  constructed-response  reading  and  writing  items  can  be  found  in  Table  7-1. 

As  with  mathematics,  the  scorers  used  the  same  training  materials  as  in  previous  assessments  for 
reading  and  writing.  Thus,  there  was  no  need  to  select  new  training  material  from  current  year  responses. 
Preparation  for  the  three  long-term  trend  scoring  projects  began  with  identifying  samples  from  previous 
years  as  indicated  below.  Scores  assigned  in  assessment  booklets  from  1984  (reading  responses)  and 
1988  (writing  responses)  had  been  masked  in  previous  years  to  ensure  that  scoring  for  training,  and 
subsequent  long-term  trend  reliability  scoring,  would  be  done  without  knowledge  of  the  previous  scores 
given.  The  1994  booklets  required  no  masking  because  scores  had  never  been  written  directly  in  the 
booklets.  Finally,  clerical  support  staff  members  matched  scoring  sheets  with  the  booklets  selected  for 
rescore  after  they  had  been  pulled  from  the  warehouse. 

The  formal  training  for  the  long-term  trend  assessments  was  divided  into  two  parts  to 
accommodate  the  reading  and  writing  items.  The  reading/writing  long-term  trend  scoring  project  started 
with  one  team  of  seven  readers  with  one  table  leader  for  fall  trend.  Six  of  the  same  scorers  and  the  same 
table  leader  continued  for  winter  and  spring  trends.  Dates  for  scoring  the  three  seasons  of  trend  booklets 
are  given  in  Table  7-2.  During  training  each  reader  received  a photocopied  packet  of  materials  used  in  the 
1984  scoring  of  the  reading  items  and  the  1988  scoring  of  the  writing  items. 

Prior  to  scoring  any  1996  reading  and  writing  trend  material,  a training  reliability  report  was 
generated  using  a 25  percent  sampling  of  the  1984  assessment  materials  for  reading  and  1988  materials 
for  writing.  Following  the  formal  training  sessions,  the  readers  scored  this  material  on  scannable  scoring 
sheets  produced  for  specific  booklet  types  with  the  appropriate  long-term  trend  items  pre-printed  on  the 
scoring  sheets.  These  sheets  were  then  routed  to  scanning  under  a special  job  number  to  ensure  that  this 
material  was  labeled  for  training  scoring  only.  The  scoring  coordinator  was  able  to  generate  a computer 
report  that  listed  the  individual  and  group  percent  agreement  by  item.  The  system  automatically 
compared  the  new  score  with  the  original  score  assigned  in  the  1984  or  1988  scoring  and  produced  a 
report  on  the  training  reliability.  T-tests  were  also  generated  for  each  item  to  verify  comparability  of 
scoring  across  years.  The  NCS  scoring  specialist  then  conferred  with  the  appropriate  ETS  staff  on  this 
training  reliability  agreement  report  before  proceeding  with  scoring. 

All  readers  for  this  project  were  experienced  scorers  with  a minimum  of  a bachelor’s  degree.  One 
team  member  had  scored  the  same  long-term  trend  items  for  the  1994  assessment.  The  team  read 
materials  as  they  arrived  at  the  scoring  center,  with  occasional  breaks  in  scoring  when  receipts  were 
slow.  The  table  leader  monitored  consistency  within  the  current  year  as  well  as  across  years  on  a daily 
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basis  as  indicated  below.  The  number  of  booklets  processed  and  constructed  responses  scored  for  each 
age  level  can  be  found  in  Table  6-1  in  Chapter  6. 

Reliability  studies  were  conducted  for  the  scoring  of  the  long-term  trend  reading  and  writing 
items.  For  the  1996  booklets,  33  percent  of  the  constructed-response  items  were  scored  by  a second 
reader  to  produce  interreader  reliability  statistics.  In  addition,  a long-term  trend  reliability  study  was 
conducted  to  ensure  that  the  scoring  procedures  were  consistent  with  those  used  in  1984,  1988,  and  1994. 
For  this  study,  350  of  the  1984  reading  responses  and  350  of  the  1988  writing  responses  were  sampled. 
Also,  250  of  both  reading  and  writing  responses  from  1994  were  sampled  for  rescore.  The  scoring  of 
these  long-term  trend  samples  was  intermixed  with  the  scoring  of  the  1996  reading  and  writing  trend 
material.  The  readers  selected  a bundle  of  approximately  five  of  each  booklet  type  each  day  and  gridded 
their  scores  on  separate  scannable  scoring  sheets  for  each  item.  These  sheets  were  then  scanned  and 
cross-referenced  with  the  original  data  tape  to  extract  information  for  long-term  trend  reliability 
reporting.  T-tests  were  generated  daily  to  verify  comparability  of  scoring  across  years.  Note  that  only 
primary  trait  scores  were  compared  in  the  across-year  rescore.  Secondary  traits  and  items  scored  for 
attemptedness  only  were  not  second  scored  in  the  current  year  nor  rescored  in  the  long-term  trend 
sample.  Composite  ranges  of  interreader  reliability  figures  can  be  found  in  Table  7-3. 


7.2.3  Holistic  Writing  Scoring 

Certain  writing  items  included  in  the  long-term  trend  assessment  were  scored  holistically. 

Holistic  scoring  is  based  on  an  overall  judgment  of  writing  fluency  and  structure.  Because  a six-point 
holistic  scale  was  used,  no  scores  fell  exactly  in  the  middle  of  the  scale.  This  was  done  to  force  readers  to 
categorize  a student’s  response  into  either  the  top  half  or  the  bottom  half  of  the  scale.  Within  the  upper 
and  lower  halves  of  the  scale,  the  scores  reflected  the  degree  to  which  the  student  demonstrated  fluency, 
structure,  or  both,  in  responding  to  the  prompt. 

In  the  1996  holistic  scoring  session,  two  items  were  scored  from  each  of  the  three  grade  levels. 
The  items  scored  at  grade  4 were 

• flashlight,  an  imaginative  task,  and 

• spaceship,  a persuasive  task. 

At  grades  8 and  1 1,  the  two  items  scored  were 

• food  on  the  frontier,  an  informative  task,  and 

• recreation  opportunities,  a persuasive  task. 

The  responses  scored  were  taken  from  six  assessment  years:  1984,  1988,  1990,  1992,  1994,  and  1996. 
Fifty-six  readers  participated  in  the  1996  holistic  scoring  session.  They  were  organized  into  seven  teams 
of  eight  readers,  with  each  team  led  by  a table  leader  and  assisted  by  one  clerical  aide. 

The  total  number  of  responses  scored,  readers,  table  leaders,  and  dates  are  given  in  Table  6-1  in 
Chapter  6.  Detailed  lists  of  responses  broken  down  by  prompt,  grade,  and  year  are  given  in  the  Report  of 
Processing  and  Professional  Scoring  Activities  (NCS,  1996). 
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7.2.3. 1  Materials  Preparation 


All  grade-eligible  student  booklets  with  on-task  scores  were  selected  from  each  of  the  six 
assessment  years  and  assembled  into  bundles  of  25  booklets  each.  Bundle  header  sheets  were  then 
generated  from  the  existing  data  file  of  student  booklet  identification  numbers.  The  system  assigned  the 
first  25  grade-eligible  booklet  ID  numbers  for  each  specific  booklet  type  to  a bundle.  The  next  25 
booklets  for  that  booklet  type  were  assigned  the  next  consecutive  bundle  number,  and  so  on  until  all 
booklets  were  assigned.  The  bundle  header  was  placed  on  top  of  the  bundle  of  booklets  and  these  were 
then  rubber-banded  together.  In  addition  to  the  bundle  number,  the  bundle  header  showed  the  name  of  the 
item  to  be  scored,  the  grade  level,  the  assessment  year,  and  the  booklet  ID  numbers  of  the  booklets  in  that 
bundle. 


For  each  bundle,  two  sets  of  scoring  sheets  were  generated — one  for  the  first  scoring  and  one  for 
the  second  scoring.  The  scoring  sheets  indicated  the  bundle  number,  the  name  of  the  item  to  be  scored, 
the  grade  level,  the  assessment  year,  and  the  booklet  ID  number.  Second  scoring  sheets  were  generated 
for  a random  25  percent  of  the  booklets  in  each  bundle.  For  both  first  and  second  scoring  sheets,  a 
separate  sheet  was  generated  for  each  booklet  in  the  bundle. 

Since  some  of  the  booklets  for  grades  8 and  1 1 contained  two  items  to  be  scored  during  the 
holistic  scoring.  Recreation  Opportunities  and  Food  on  the  Frontier,  two  separate  bundle  headers  and  sets 
of  scoring  sheets  were  created  for  these  booklets  to  accommodate  the  two  scorings  of  the  same  booklets. 
The  appropriate  header  and  corresponding  scoring  sheets  for  both  items  were  placed  with  the  booklet 
bundle  prior  to  scoring.  Upon  completion  of  the  holistic  scoring  for  the  Food  on  the  Frontier  item, 
clerical  staff  removed  the  header  and  scoring  sheets  for  this  item  and  replaced  them  with  the  sheets  for 
the  Recreation  Opportunities  item;  then  the  responses  for  the  Recreation  Opportunities  item  were  scored. 

All  bundles  were  separated  by  item  on  carts.  A sheet  identifying  the  item  and  year  was  attached 
to  each  cart.  Thus,  all  bundles  could  be  readily  identified  and  retrieved  quickly  and  easily. 


7.2.3.2  Training  Preparation 

Three  days  prior  to  the  beginning  of  holistic  scoring,  the  ETS  writing  coordinator  met  with  the 
table  leaders  and  NCS  administrative  assistant  to  prepare  training  materials.  They  first  reviewed  the 
training  materials  from  the  1994  scoring  session  and  augmented  them  with  sample  responses  from  the 
1995-96  assessment.  The  training  packets  included  a.  familiarization  and  range  finder  set  each  containing 
six  examples,  an  upper  half  and  a lower  half  set  each  containing  three  examples,  and  three  calibration 
sets  each  containing  four  examples.  The  sets,  together  with  their  scoring  guides  and  keys,  can  be  found  in 
the  Report  of  Processing  and  Professional  Scoring  Activities  (NCS,  1996). 

Following  standard  ETS  procedures,  the  training  of  the  readers  was  conducted  immediately  prior 
to  the  beginning  of  scoring  for  each  individual  item. 


7.2.3.3  Holistic  Scoring  Training 

The  training  materials  for  the  first  item  were  distributed  to  each  reader.  Each  reader  read  a copy 
of  the  prompt  and  the  holistic  scoring  guide  for  that  item.  The  scoring  guide  was  explained  by  the  writing 
long-term  trend  coordinator.  A clerical  aide  was  assigned  to  each  of  the  seven  teams  to  keep  the  materials 
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flowing  in  an  organized  and  efficient  manner.  Training  began  on  June  iO,  1996.  The  general  sequence  of 
training  for  each  of  the  six  prompts  was  as  follows: 

Scoring  Guide  and  Prompt  - The  scoring  guide  was  introduced  and  briefly  discussed, 
followed  by  a reading  of  the  prompt  and  a general  discussion  of  the  scoring  expectations. 

Familiarization  Set  - Readers  were  asked  to  arrange  the  papers  in  the  familiarization  set 
into  ‘best  to  worst’  order,  assigning  a score  to  each  paper,  with  no  score  being  used  more 
than  once.  The  writing  long-term  trend  coordinator  announced  the  scores  given  by  the 
table  leaders  for  each  of  the  individual  papers  in  the  set.  The  table  leaders  discussed  with 
their  teams  the  rationale  behind  these  scores,  using  the  scoring  guide. 

Range  Finders  - The  writing  long-term  trend  coordinator  read  the  six  range  finders  (best 
to  worst)  and  discussed  the  rationale  behind  the  scores  given.  If  there  was  too  much 
discussion  regarding  ‘why,’  an  alternate  range  finder  for  that  score  point  was  inserted 
later.  It  was  at  this  point  the  writing  long-term  trend  coordinator  asked  the  readers  to 
look  only  to  their  range  finders  and  no  longer  to  the  scoring  guide. 

Upper  Half  - Readers  first  scored  three  papers  representing  the  upper  half.  There  was  a 
sample  of  a four-,  a five-,  and  a six-score  point  paper  included.  A tally  was  kept  of  the 
number  of  readers  assigning  each  score  point.  After  the  tally,  the  writing  long-term  trend 
coordinator  announced  the  scores  assigned  in  the  preparation  process  and  compared  them 
to  the  scores  on  the  tally  sheet. 

Lower  Half  - The  readers  were  then  asked  to  score  three  lower-half  papers.  This  sample 
included  a one-,  a two-,  and  a three-score  point  paper.  Again,  a tally  of  the  scores 
assigned  by  the  individual  readers  was  kept.  The  table  leaders  discussed  the  rationale 
behind  the  reader-assigned  scores  including  both  the  upper  and  lower  half  score  point 
papers  at  this  time  and  answered  any  team  member  questions. 

Calibration  Set  1 - Prior  to  the  beginning  of  scoring,  the  writing  long-term  trend 
coordinator  assigned  Calibration  Set  1 consisting  of  four  papers.  After  scoring 
Calibration  Set  1,  a tally  was  kept  and  the  writing  long-term  trend  coordinator  announced 
the  scores.  The  table  leader  and  team  discussed  the  readers’  scores  and  questions. 

Calibration  Sets  2 and  3 -.  A calibration  set  was  used  after  a break  of  longer  than  15 
minutes  (lunch,  morning  break,  afternoon  break,  or  overnight).  The  readers  reviewed  the 
range  finders  and  the  table  leaders  provided  a one-sentence  review  of  each  of  the  score 
points.  The  readers  scored  the  set  of  four  papers  and  the  scores  given  were  tallied.  The 
tallies  were  kept  for  each  of  these  calibration  sets  and  can  be  found  with  the  training 
materials  in  the  Report  of  Processing  and  Professional  Scoring  Activities  (NCS,  1996). 


7.2.3.4  Holistic  Scoring 


Holistic  scoring  occurred  during  the  week  of  June  10,  1996.  When  scoring  for  an  item  began, 
each  reader  was  given  a bundle  to  score.  The  reader  entered  his  or  her  reader  ID  number  in  the 
appropriate  space  on  each  scoring  sheet  and  scored  all  the  responses  in  the  bundle.  The  readers  were 
directed  to  check  the  booklet  number  listed  on  the  scoring  sheet  against  the  booklet  ID  number  printed  on 
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the  front  of  the  assessment  booklet.  If  any  discrepancies  were  found,  they  were  brought  to  the  table 
leader  for  resolution. 

When  the  first  scoring  of  a bundle  of  booklets  was  completed,  the  reader  handed  the  first  scoring 
sheets  to  the  clerical  support  staff  who  placed  the  bundle  of  booklets,  with  the  bundle  header  and  second 
scoring  sheet  still  on  top,  back  on  the  designated  cart.  NCS  clerical  staff  distributed  all  bundles  to  the 
readers  to  ensure  an  even  flow  of  material  and  a distribution  of  second  scoring  among  teams. 

Reliability  scoring  was  handled  in  the  same  manner  as  first  scoring.  When  the  second  scoring 
was  completed,  the  second  scoring  sheets  were  placed  on  the  table  leader’s  desk.  All  bundles  of  booklets 
were  placed  on  carts  and  removed  from  the  scoring  area.  Clerical  staff  then  sorted  all  scoring  sheets  and 
routed  them  to  the  scanning  area  to  be  entered  onto  the  database. 

The  table  leaders  read  through  each  reader’s  entire  initial  bundle  of  booklets  and  evaluated  the 
scores  assigned  by  the  reader.  This  process  is  known  as  backreading.  They  also  periodically  backread  the 
remaining  bundles  throughout  the  reading  of  each  item.  If  discrepant  scoring  occurred,  the  table  leader 
brought  it  to  the  attention  of  the  reader.  If  the  problem  recurred,  the  reader  was  retrained  by  the  table 
leader.  If  the  problem  continued,  the  writing  long-term  trend  coordinator  assisted  with  the  re-training. 

Twenty-five  percent  of  the  papers  were  read  again  by  a reader  from  a different  team.  A 
preliminary  reliability  measure  was  taken  by  the  administrative  assistant  approximately  two  to  three 
hours  after  the  beginning  of  scoring. 

Scoring  the  booklets  took  between  four  and  seven  hours  depending  on  the  grade  level  and  the 
type  of  writing.  The  readers  were  trained  and  scored  the  items  in  the  following  order: 

Informative:  Food  on  the  Frontier Grade  1 1 , followed  by  Grade  8 

Persuasive:  Recreation  Opportunities Grade  1 1 , followed  by  Grade  8 

Spaceship Grade  4 

Narrative:  Flashlight ..Grade  4 

The  scoring  was  completed  on  Sunday,  June  16,  1996.  A more  detailed  log  of  daily  activities  can  be 
found  in  the  Report  of  Processing  and  Professional  Scoring  Activities  (NCS,  1996). 


Table  7-4 

Holistic  Writing  Scoring  Reliability  Figures 


Grade 

Item 

Exact  Agreement 

Adjacent  Agreement 

4 

Flashlight 

52.1% 

89.6% 

Spaceship 

58.3% 

93.5% 

8 

Food  on  the  Frontier 

53.1% 

94.5% 

Recreational  Opportunities 

58.2% 

95.3% 

11 

Food  on  the  Frontier 

56.3% 

94.5% 

Recreational  Opportunities 

57.1% 

94.9% 

Total 

56.1% 

94.0% 

After  all  the  score  sheets  were  scanned,  the  scanner  operator  produced  a final  report  showing  the 
n-counts  scored  and  the  interreader  agreement  rates.  The  figures  are  given  at  the  item  level  in  Table  7-4. 


0 


175 


They  are  given  in  more  detail,  broken  down  by  prompt,  grade,  and  year,  in  the  Report  of  Processing  and 
Professional  Scoring  Activities  (NCS,  1996). 

7.2.4  Writing  Mechanics 

Mechanics  scoring  focused  on  the  extent  to  which  the  writer  can  control  the  conventions  of 
written  English — grammar,  spelling,  capitalization  and  punctuation.  In  addition,  the  procedures  include 
identifying  sentence  structures  and  word  choice  errors.  A team  of  33  readers  and  six  table  leaders  scored 
selected  essays  for  each  age/grade  group  from  the  writing  long-term  trend  assessments  conducted  in  years 
1988,  1990,  1992,  1994,  and  1996.  Spaceship,  an  imaginative  task,  was  scored  at  age  9/grade  4 and 
recreation  opportunities,  a persuasive  task,  was  scored  at  both  age  13/grade  9 and  age  17/grade  11. 

7.2.4.1  Preparation 

Essays  to  be  scored  from  the  1996  assessment  were  selected  according  to  ETS  specifications  as 
follows.  For  each  booklet  in  which  each  item  appeared: 

1.  select  all  grade-eligible  booklets  with  primary  trait  scores  greater  than  0 and  less 
than  7,  in  other  words  only  on-task  responses; 

2.  select  every  third  booklet;  and 

3.  select  all  Black  students  (based  on  the  student’s  response  in  the  background 
questions)  not  picked  in  Step  2. 

As  a result  of  this  selection  process,  1,593  essays  were  scored  from  the  1996  assessment.  In  addition,  10 
percent  of  the  essays  previously  scored  for  mechanics  from  the  1988,  1990,  1992,  and  1994  assessments 
were  rescored  for  reliability.  This  sample  was  selected  by  locating  specific  booklets  from  a list  generated 
by  ETS  and  resulted  in  a rescore  of  736  essays  for  all  four  years. 

In  preparation  for  the  scoring  process,  copies  were  made  of  each  selected  essay  and  its 
corresponding  booklet  cover.  The  booklet  cover,  containing  assessment  year,  age/grade,  primary 
sampling  unit  (PSU)  and  student  ID  information,  was  stapled  to  the  essay.  Papers  were  then  grouped  by 
assessment  year  and  grade  into  packets  of  20.  Packets  were  numbered  consecutively  and  were  identified 
by  headers.  Three  identical  sets  of  packets  (A,  B,  and  C)  were  assembled  since  each  essay  had  to  be 
scored  independently  by  two  readers  and  discrepancies  had  to  be  resolved  by  a table  leader.  Packets  A 
and  B were  used  by  the  two  readers  and  Packet  C was  used  by  the  resolver.  For  prior  year  scoring,  the 
same  sample  was  used  as  in  1994.  Therefore,  copies  were  made  from  the  1994  master  sets  rather  than 
returning  to  the  original  booklets.  The  master  sets  were  labeled  as  D Packets  and  warehoused  for 
potential  future  use. 

7.2.4.2  Writing  Mechanics  Training 

Training  of  the  six  table  leaders  and  33  readers  was  conducted  by  the  writing  long-term  trend 
coordinator  during  the  week  of  July  15,  1996.  The  training  involved  a detailed  discussion  of  the  scoring 
guide.  The  writing  coordinator  presented  the  main  sections  of  the  guide: 

• type  of  sentence  construction, 

• faulty  sentence  construction, 

• punctuation,  and 

• word  level  categorization. 
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Copies  of  pertinent  resource  information  were  distributed,  briefly  reviewed,  and  reference  materials  were 
identified.  After  discussion  of  each  of  the  main  sections  of  the  guide,  the  group  reviewed  the  scored 
papers  from  the  training  packet,  paying  special  attention  to  scores  reflecting  the  category  under 
discussion. 

To  further  train  the  readers,  the  trainer  used  a pool  of  responses  from  thel988  and  1990 
assessments  that  has  been  scored  for  mechanics  in  1990  but  not  used  in  the  rescore  sample  in  1996. 
Copies  were  made  of  these  training  essays  to  be  used  for  practice.  Each  reader  then  individually  scored  a 
selected  group  of  essays.  The  scores  were  compared  among  the  group,  discussions  were  held  when 
discrepancies  occurred,  and  again  references  were  made  to  resource  materials  or  to  the  scored  sample 
papers.  When  the  group  was  comfortable  with  the  decisions  being  made,  the  actual  scoring  began.  Copies 
of  these  sets  and  the  scoring  guidelines  can  be  found  in  the  Report  of  Processing  and  Professional 
Scoring  Activities  (NCS,  1996). 


7.2.4.3  Scoring 

The  actual  scoring,  resolution,  data  entry  work  and  proofreading  of  writing  mechanics  began  on 
July  18  and  was  completed  on  July  31,  1996.  In  selecting  packets  for  scoring,  readers  alternated  among 
the  different  grade  levels  and  assessment  years.  The  mechanics  readers  marked  each  paper  with  a series 
of  symbols,  addressing  the  elements  of  sentence  type,  sentence  construction,  word  choice,  spelling, 
punctuation,  and  capitalization.  These  symbols,  written  in  red  ink,  designated  each  word  or  punctuation 
mark  in  error  and  indicated  sentence  type  or  faulty  sentence  construction.  Each  essay  was  scored 
independently  by  two  different  readers  selecting  either  Packet  A or  B. 

To  track  the  movement  of  the  packets,  the  NAEP  internal  tracking  log  was  used.  As  readers  and 
resolvers  worked  on  particular  packets,  the  appropriate  columns  were  initialed  and  dated.  This  enabled 
NCS  staff  to  see  at  a glance  the  status  of  each  packet.  The  completed  tracking  logs  have  been  warehoused 
with  the  training  materials.  A sample  of  a blank  tracking  log  can  be  found  in  the  Report  of  Processing 
and  Professional  Scoring  Activities  (NCS,  1996). 

Resolution  and  quality  control  were  conducted  by  table  leaders  who  compared  the  scores  marked 
on  copies  A and  B of  each  unique  packet  and  resolved  any  discrepancies.  After  determining  the 
appropriate  marks,  the  resolver  used  the  unmarked  copy  C to  record  the  final  version.  The  copy  with  the 
resolved  marks  was  sent  to  the  word  processing  area  in  NCS’s  Creative  Services  department  for 
transcription  as  described  below.  To  avoid  confusion,  unused  copies  were  discarded  or  returned  to 
original  readers  with  feedback  information  and  with  follow-up  training,  if  deemed  necessary. 

To  maintain  a consistent  scoring  standard,  the  six  table  leaders,  along  with  the  NCS  performance 
assessment  specialist,  met  twice  daily  to  resolve  any  questions  that  arose  during  the  scoring.  Resolution 
scoring  allowed  the  table  leaders  to  determine  the  accuracy  of  individual  readers.  If  a reader  was 
confused  by  a facet  of  the  coding,  the  table  leader  would  approach  the  reader  individually.  If  the  table 
leaders  identified  a trend  in  the  coding,  the  issue  would  be  broached  at  the  re-calibration  sessions.  Twice 
a week,  for  approximately  30  minutes  on  Wednesday  mornings  and  Friday  afternoons,  the  six  tables 
stopped  scoring  and  re-calibrated;  table  leaders  brought  up  pertinent  scoring  issues  and  readers  asked  for 
clarification  on  coding  decisions.  , ' 

Resolved  packets  were  sent  to  the  NCS  Creative  Services  department  where  the  text  of  the 
essays,  along  with  the  assigned  marks  and  identification  information,  were  entered  into  a data  file.  Essays 
were  typed  exactly  as  they  were  written.  Each  essay  was  typed  on  a separate  page  and  double-spaced 
pages  were  printed  for  proofreading.  The  scoring  team  proofread  the  data  entry  work  against  the  scored 
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papers.  Student  identification  and  PSU  numbers  were  checked  and  discrepancies  were  resolved. 
Corrections  were  indicated  in  red  on  the  typed  copy.  If  corrections  were  needed,  all  pages  belonging  to  a 
packet  were  returned  to  the  Creative  Services  department  for  additional  corrections.  Complete  and 
correct  packets  were  uploaded  to  the  NCS  mainframe.  The  data  were  reformatted  according  to  ETS 
specifications  and  a data  file  containing  the  scored  information  was  sent  to  ETS  on  August  7,  1996. 


7.3  MAIN  NAEP  ASSESSMENT 
7.3.1  Selection  of  Training  Papers 

A pool  of  papers  to  be  used  for  training  for  the  NAEP  main  assessments  was  selected  by  NCS  in 
February  and  March  of  1996.  Persons  identified  as  potential  mathematics  and  science  table  leaders  were 
selected  to  copy  student  responses.  Team  leaders,  with  assistance  of  the  potential  table  leaders,  gave 
tentative  scores  to  the  responses  and  selected  50  student  responses  from  each  dichotomous  item,  75 
responses  from  each  3-point  item,  100  responses  from  each  4-point  item  and  125  from  each  5-point  item. 
Because  NCS  staff  screened  the  responses,  the  pool  sent  to  the  ETS  test  development  specialists 
contained  a full  range  of  point  values. 

NCS  staff  numbered  the  papers  sequentially  and  copied  the  sets.  NCS  retained  and  filed  the 
originals  and  sent  the  copies  to  the  appropriate  subject  area  coordinators  at  ETS.  ETS  returned  a list  of 
the  anchor  and  practice  sets  with  scores  to  NCS  st^f,  who  used  the  master  file  copies  to  create  training 
sets.  NCS  staff  then  masked  the  sequential  reference  numbers  and  wrote  the  actual  scores  on  the  anchor 
papers  and  a new  sequential  reference  number  for  the  training  sets. 

The  NCS  copy  center  and  PSC  clerical  staff  shared  responsibility  for  making  multiple  copies  of 
the  sets  for  scorers.  The  master  sets,  team  leader/trainer  copy  and  table  leader  copies  also  had  keys  to  the 
training  sets.  When  copying  was  complete,  the  master  copy  was  placed  in  the  appropriate  file. 


7.3.2  Calibration  Policies 

During  scoring,  the  teams  used  calibration  sets  to  calibrate  on  a daily  basis  and  to  calibrate 
across  longer  periods  of  time.  The  table  leader  built  pools  of  items  for  calibration,  which  were  then 
distributed  to  the  scorers  in  sets  of  five  or  ten,  depending  upon  the  complexity  of  the  item,  whenever  a 
break  of  longer  than  15  minutes  occurred,  such  as  after  lunch  or  at  the  beginning  of  a new  scoring  day. 
All  readers  on  the  team  scored  the  same  calibration  sets,  and  the  system  compared  the  scores  of  each 
reader  to  the  scores  assigned  by  the  trainer  and  table  leader.  The  table  leader  reviewed  the  interreader 
agreement  report  with  the  trainer  and  the  ETS  subject  area  coordinator,  discussed  any  discrepancies  that 
arose,  and  then  proceeded  with  scoring. 

Whenever  a team  returned  to  scoring  an  item  after  having  worked  on  a different  item  in  the 
meantime,  the  team  scored  a calibration  set  of  75  responses  and  analyzed  the  results  before  proceeding. 
This  occurred  in  mathematics  because  many  mathematics  items  were  scored  in  two  sessions  since  the 
first  sweep  through  the  items  was  done  while  booklets  were  still  arriving  from  the  field.  If  the  item  had 
fewer  than  500  responses  left  out  of  a pool  that  contained  both  main  and  state  samples,  an  extended 
calibration  was  waived.  In  science,  the  teams  did  not  begin  scoring  most  items  until  all  responses  to  the 
item  were  available  for  scoring,  so  this  type  of  calibration  was  not  necessary. 
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Table  leaders  printed  and  archived  hard  copies  of  all  calibration  sets  used  for  scoring.  For  more 
information  on  the  functionality  of  the  calibration  tool,  see  Section  1.3.33. 


7.3.3  Table  Leader  Utilities 

Two  of  the  significant  advantages  of  the  image-scoring  system  were  the  ease  of  regulating  the 
flow  of  work  to  readers  and  the  ease  of  monitoring  scoring.  The  image  system  provided  table  leaders 
with  tools  to  determine  reader  qualification,  to  backread  scores,  to  determine  reader  calibration,  to 
monitor  interreader  reliability,  and  to  gauge  the  rate  at  which  scoring  was  being  completed.  These 
various  tools  are  described  below. 


7.3.3. 1 Reader-Qualiflcation  Tool 

One  of  the  utilities  at  a table  leader’s  disposal  was  a qualification  algorithm  used  after  training 
on  extended  constructed-response  items.  The  table  leader  would  give  identical  qualification  packets  to 
each  reader.  These  packets  contained  10  student  responses  to  be  independently  scored  by  the  readers. 
After  the  readers  finished,  the  table  leader  would  enter  each  reader’s  scores  into  the  computer  for 
tabulation.  The  computer  would  calculate  each  reader’s  percentage  of  exact,  adjacent,  and  non-adjacent 
agreement  with  the  master  key.  If  a reader  attained  a percentage  of  exact  agreement  above  a pre- 
determined threshold  of  80%,  the  reader  would  be  allowed  to  score.  Readers  not  attaining  the  pre- 
determined threshold  were  handled  on  a case-by-case  basis — typically  receiving  individual  training  by 
the  trainer  or  the  NCS  table  leader  before  being  allowed  to  score.  A table  leader  could  cancel  a reader’s 
qualification  to  score  an  item  if  review  of  a reader’ s work  indicated  inaccurate  scoring  and  that 
supplemental  training  was  necessary.  Note  that  reader  qualification  was  required  only  on  extended 
constructed-response  items  involving  4 or  more  score  point  levels. 


7.3.3.2  Backreading  Tool 

After  scoring  began,  NCS  table  leaders  reviewed  each  reader’s  progress  using  a backreading 
utility  that  allowed  the  table  leader  to  review  papers  scored  by  each  reader  on  the  team.  Typically,  a table 
leader  reviewed  responses  scored  by  each  reader  in  quantities  similar  to  the  amount  second  scored  (i.e., 
6%  for  items  with  both  state  and  national  samples  and  more  for  items  with  only  a national  sample).  Table 
leaders  noted  the  score  the  reader  awarded  each  response  as  well  as  the  score  a second  reader  gave  that 
same  response.  This  was  done  as  an  interreader  reliability  check.  Alternatively,  a table  leader  could 
choose  to  review  all  student  responses  given  a particular  score  to  determine  if  the  team  as  a whole  was 
scoring  consistently.  Both  of  these  review  methods  used  the  same  display  screen  and  showed  the  ID 
number  of  the  reader  and  the  scores  awarded.  If  the  table  leader  disagreed  with  the  score  given  an  item, 
he  or  she  discussed  it  with  the  reader  for  possible  correction.  Replacement  of  scores  by  the  table  leader 
was  done  only  with  the  knowledge  and  approval  of  the  reader,  thereby  serving  as  a learning  experience 
for  the  reader.  Additionally,  neither  score  was  changed  in  the  case  where  the  response  was  second 
scored. 


7.3.3.3  Calibration  Tool 

While  backreading,  a table  leader  could  identify  individual  responses  for  inclusion  in  a pool  of 
calibration  papers.  These  papers  could  be  selected  because  they  exemplified  criteria  set  down  in  the 
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scoring  rubrics  or  because  they  were  unusual  and  pointed  out  less  obvious  aspects  of  the  scoring 
guidelines.  After  selecting  a number  of  papers  for  inclusion  in  the  calibration  set,  the  table  leader  could 
decide  to  route  any  number  of  these  calibration  papers  to  the  scorers.  A typical  number  of  papers  routed 
to  scorers  during  a mid-scoring  calibration  was  10,  although  the  image  system  could  accommodate  as 
many  or  as  few  as  the  table  leader  and  trainer  determine  necessary  to  check  the  accuracy  of  scoring. 
When  all  scorers  had  completed  the  calibration  set,  the  table  leader  could  then  produce  an  interteader 
reliability  report  on  the  scoring  of  the  calibration  set. 


7.3.3.4  Tool  for  Monitoring  Interreader  Reliability 

During  the  scoring  of  an  item  or  the  scoring  of  a calibration  set,  the  table  leader  could  monitor 
progress  using  an  interteader  reliability  tool.  This  display  tool  could  be  used  in  either  of  two  modes: 

1.  to  display  information  of  all  first  readings  versus  all  second  readings,  or 

2.  to  display  all  readings  of  an  individual  that  were  also  scored  by  another  reader  versus 
the  scores  assigned  by  the  other  readers. 

The  information  was  displayed  as  a matrix  with  scores  awarded  during  first  readings  displayed  in  rows 
and  scores  awarded  during  second  readings  displayed  in  columns  for  Mode  1 and  the  individual’s  scores 
in  rows  and  all  other  readers  in  columns  for  Mode  2.  In  this  format,  instances  of  exact  agreement  fell 
along  the  diagonal  of  the  matrix.  For  completeness,  data  in  each  cell  of  the  matrix  contained  the  number 
and  percentage  of  cases  of  agreement  (or  disagreement).  The  display  also  contained  information  on  the 
total  number  of  second  readings  and  the  overall  percentage  of  exact  agreement  on  the  item.  Since  the 
interreader  reliability  reports  were  cumulative,  a printed  copy  of  the  exact  agreement  of  each  item  was 
made  every  day  and  compared  to  previously  generated  reports. 


7.3.3.S  Tool  for  Monitoring  the  Rate  of  Scoring 

The  table  leaders  were  able  to  monitor  work  flow  using  a status  tool  that  displayed  the  number  of 
items  scored,  the  number  of  items  first-scored  that  still  needed  to  be  second-scored,  the  number  of  items 
remaining  to  be  second-scored,  and  the  total  number  of  items  remaining  to  be  scored.  This  allowed  the 
team  leaders  and  performance  assessment  specialists  to  accurately  monitor  the  rate  of  scoring  and  to 
estimate  the  time  needed  for  completion  of  the  various  phases  of  scoring. 


7.3.3.6  Scoring  Buttons 

To  assign  a score,  readers  clicked  the  mouse  over  a button  contained  in  the  scoring  window. 
Since  buttons  were  included  only  for  valid  scores,  there  is  no  need  to  edit  for  out-of-range  scores. 
Another  recent  development  was  the  implementation  of  a tool  that  allowed  the  performance  assessment 
specialist  to  label  scoring  buttons  with  key  phrases,  correct  responses  or  certain  incorrect  responses  that 
were  to  be  tracked.  This  enhanced  scoring  as  readers  no  longer  had  to  mentally  translate  a student’s 
response  into  a numerical  value  before  choosing  a scoring  button  on  the  image  screen. 
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7.3.4  Main  Mathematics  Assessment 


The  mathematics  portion  of  the  1996  main  assessment  included  a total  of  226  discrete 
constructed-response  items.  Table  7-1  shows  the  types  and  number  of  constructed  responses  for  the 
mathematics  assessment.  A variety  of  constructed-response  items  were  used  to  measure  different 
elements  of  students’  mathematical  sophistication  and  understanding.  These  items  were  administered  in 
scannable  assessment  booklets.  The  bilingual  booklets  were  key-entry  documents.  The  items  scored 
included  traditional  computational  items,  short-answer  constructed  responses,  extended  constructed 
responses,  diagrams,  geometric  figures,  and  graphs.  Each  constructed-response  item  had  a unique  scoring 
guide  that  identified  the  range  of  possible  scores  for  the  item  and  defined  the  criteria  to  be  used  in 
evaluating  student  responses.  Long-term  trend  items  that  focused  on  the  students’  computational  ability 
were  typically  scored  on  a right/wrong  basis.  The  scoring  guides  for  the  more  complex  items  were 
developed  to  be  of  diagnostic  value,  by  including  categories  that  reflected  partial  credit  and/or  different 
kinds  of  incorrect  answers  that  indicated  particular  misunderstandings.  New  items  developed  for  the 
1993  and  1995  NAEP  field  tests  were  scored  on  a partial-credit  scale  (3-point  or  4-point)  or  on  an 
extended  scale  (5-point). 

The  operational  assessment  included,  for  the  first  time  other  than  the  field  test,  blocks  that  were 
based  on  a theme.  There  were  two  different  theme  blocks  administered  at  each  grade.  General 
information  on  the  number  of  constructed  responses  scored  can  be  found  in  Table  6-1  in  Chapter  6.  Table 
7-5  gives  more  detailed  information  by  grade  and  booklet  type  (spiral,  estimation,  theme,  and  advanced). 


7.3.4. 1 Training 

The  training  for  each  mathematics  item  was  conducted  by  mathematics  specialists  from  ETS  and 
NCS  just  prior  to  the  scoring  of  that  item.  Training  and  scoring  began  on  Wednesday,  March  13,  and 
ended  on  Monday,  May  6,  1996.  The  NCS  mathematics  performance  assessment  specialist  and  selected 
NCS  team  leaders  conducted  all  the  training  of  the  short-term  trend  items  scored  in  March.  The  ETS 
mathematics  coordinator  met  with  each  trainer  individually  before  the  beginning  of  training  to  discuss 
any  questions  and/or  discrepancies.  He  returned  to  NCS  at  the  beginning  of  April  to  review  individually 
the  items  assigned  in  April.  The  NCS  training  staff  added  another  team  leader  to  assist  in  training  for  the 
evening  shift  in  April. 

Training  involved  explaining  the  item  and  its  scoring  guide  and  discussing  responses  that 
represented  the  various  score  points  in  the  guide.  When  this  was  completed,  the  readers  scored  and 
discussed  from  5 to  35  selected  “practice  papers”  for  each  item.  Next,  readers  practiced  scoring  by 
gathering  around  a single  image  terminal  and  scoring  several  responses  to  the  item.  Once  the  trainer  and 
the  table  leader  determined  the  individuals  on  the  team  understood  the  scoring  guide,  the  table  leader 
qualified  the  scorers  to  enter  the  system  using  the  reader-qualification  tool,  discussed  in  Section  7.3.3. 1. 


Table  7-5 

Mathematics  Constructed  Responses  Scored 


Type 

Data 

1990 

1992 

1996 

Grand 

Total 

Assessment 
Proportions 
1996  1996 

National  State 

Grade  4 

Regular 

Unique  items 

12 

36 

64 

112 

7.9% 

92.1% 

Responses  first  scored 

0 

0 

2,037,866 

2,037,866 

160,584 

1,877,282 

Responses  second  scored 

8,988 

27,031 

122,272 

158,291 

9,635 

112,637 

First  and  second  scored 

8,988 

27,031 

2,160,138 

2,196,157 

170,219 

1,989,919 

Average  % exact  agreement 

95.9 

93.8 

96.8 

95.7 

Estimation 

Unique  items 

0 

0 

3 

3 

Responses  first  scored 

0 

0 

7,317 

7,317 

Responses  second  scored 

0 

0 

439 

439 

First  and  second  scored 

0 

0 

7,756 

7,756 

Average  % exact  agreement 

N/A 

N/A 

93.3 

93.3 

Theme 

Unique  items 

0 

0 

13 

13 

Responses  first  scored 

0 

0 

23,800 

23,800 

Responses  second  scored 

0 

0 

5,950 

5,950 

First  and  second  scored 

0 

0 

29,750 

29,750 

Average  % exact  agreement 

N/A 

N/A 

95.8 

95.8 

Total 

Unique  items 

12 

36 

80 

128 

Responses  first  scored 

0 

0 

2,068,983 

2,068,983 

Responses  second  scored 

8,988 

27,031 

128,661 

164,680 

First  and  second  scored 

8,988 

27,031 

2,197,644 

2,233,663 

Average  % exact  agreement 

95.9 

93.8 

96.5 

95.7 

Average  % exact  agreement 

95.9 

93.8 

96.5 

95.7 

Grade  8 

Regular 

Unique  items 

18 

44 

69 

131 

9.3% 

90.7% 

Responses  first  scored 

0 

0 

1,991,682 

1,991,682 

184,683 

1,806,999 

Responses  second  scored 

13,374 

33,077 

119,501 

165,952 

11,081 

108.420 

First  and  second  scored 

13,374 

33,077 

2,111,183 

2,157,634 

195,764 

1,915,419 

Average  % exact  agreement 

95.3 

94.4 

96.6 

95.7 

Estirnation 

Unique  items 

0 

0 

6 

6 

Responses  first  scored 

0 

0 

13,622 

13,622 

Responses  second  scored 

0 

0 

3,405 

3,405 

First  and  second  scored 

0 

0 

17,027 

17,027 

Average  % exact  agreement 

N/A 

N/A 

97.3 

97.3 

Theme 

Unique  items 

0 

0 

13 

13 

Responses  first  scored 

0 

0 

27,690 

27,690 

Responses  second  scored 

0 

0 

6,923 

6,923 

First  and  second  scored 

0 

0 

34,613 

34,613 

Average  % exact  agreement 

N/A 

N/A 

93.9 

93.9 

(continued) 
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Table  7-5  (continued) 

Mathematics  Constructed  Responses  Scored 


Type 

Data 

1990 

1992 

1996 

Grand  Total 

Assessment 
Proportions 
1996  1996 

National  State 

Grade  8 

Advanced 

Unique  items 

0 

0 

14 

14 

Responses  first  scored 

0 

0 

33,354 

33,354 

Responses  second  scored 

0 

0 

8,339 

8,339 

First  and  second  scored 

0 

0 

41,693 

41,693 

Average  % exact  agreement 

N/A 

N/A 

95.1 

95.1 

Total 

Unique  items 

18 

■ 44 

102 

164 

Responses  first  scored 

0 

0 

2,066,348 

2,066,348 

Responses  second  scored 

13,374 

33,077 

138,168 

184,619 

First  and  second  scored 

13,374 

33,077 

2,204,516 

2,250,967 

Average  % exact  agreement 

95.3 

94.4 

96.1 

95.5 

Grade  12^ 

Regular 

Unique  items 

21 

44 

73 

138 

N/A 

N/A 

Responses  first  scored 

0 

0 

212,774 

212,774 

N/A 

N/A 

Responses  second  scored 

15,225 

32,387 

12,766 

60,378 

N/A 

N/A 

First  and  second  scored 

15,225 

32,387 

225,540 

273,152 

N/A 

N/A 

Average  % exact  agreement 

94.8 

92.1 

95.6 

94.4 

Theme 

Unique  items 

0 

0 

12 

12 

Responses  first  scored 

0 

0 

23,368 

23,368 

Responses  second  scored 

0 

0 

5,842 

5,842 

First  and  second  scored 

0 

0 

29,210 

29,210 

Average  % exact  agreement 

N/A 

N/A 

94.5 

94.5 

Advanced 

Unique  items 

0 

0 

15 

15 

Responses  first  scored 

0 

0 

44,881 

44,881 

Responses  second  scored 

0 

0 

11,220 

11,220 

First  and  second  scored 

0 

0 

56,101 

56,101 

Average  % exact  agreement 

N/A 

N/A 

95.1 

95.1 

Total 

Unique  items 

21 

44 

100 

165 

Responses  first  scored 

0 

0 

281,022 

281,022 

Responses  second  scored 

15,225 

32,387 

29,829 

77,441 

First  and  second  scored 

15,225 

32,387 

310,851 

358,463 

Average  % exact  agreement 

94.8 

92.1 

95.4 

94.5 

Grand  Total 

- Unique  items 

51 

124 

282 

457 

' 4th  grade  mathematics  had  no  advanced  booklets, 

^ 1 2th  grade  estimation  block  had  no  constructed-response  items. 
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7.3.4.2  Scoring 


Mathematics  scoring  took  place  in  two  segments  over  two  shifts  as  outlined  in  Table  7-2.  Most 
dichotomously  scored  short-term  trend  items  were  scored  in  Segment  1,  with  the  remainder  of  the  items 
scored  in  Segment  2. 

During  scoring,  the  team  leaders  continued  to  compile  notes  on  scoring  decisions  for  the  readers’ 
reference  and  guidance.  Additionally,  table  leaders  closely  monitored  interreader  reliability  using  both 
team  and  individual  statistics  as  a reference.  Consistently  throughout  the  scoring  of  each  item,  the  table 
leaders  also  performed  backreading  duties  in  which  they  reviewed  a sample  of  the  responses  scored  by 
each  reader  on  the  team.  Lead  scorers  selected  for  their  experience  and  accuracy  in  scoring  assisted  the 
table  leaders  in  backreading.  The  team  leaders  and  performance  assessment  specialist  continuously 
monitored  the  progress  of  each  team  and  noted  all  scoring-related  decisions  to  ensure  that  training  and 
scoring  progressed  smoothly  and  in  a timely  manner. 

The  codes  that  were  used  for  unscorable  mathematics  items  were: 

0 = Blank  or  random  marks 

8 = Completely  crossed-out  or  erased . 

9 = “I  don’t  know,”  refusal,  off-task,  illegible  or  language  other  than  English 


7.3.4.3  Reliability 

A minimum  of  25  percent  of  the  mathematics  responses  for  items  involved  only  in  the  national 
sample  and  6 percent  of  the  responses  for  items  involved  in  the  state  and  national  samples  were  scored  by 
a second  reader  to  obtain  statistics  on  interreader  reliability.  Responses  were  automatically  routed  for 
second  scoring  so  that  the  reader  could  not  discern  any  difference  between  a response  being  presented  for 
first  or  second  scoring.  The  reliability  figures  were  available  to  the  table  leader  as  soon  as  scoring  began 
and  could  be  viewed  on  demand.  These  figures  included  a frequency  distribution  of  all  second  scores  for 
the  team  and  a frequency  distribution  for  all  second  scores  for  individual  members  of  the  team  who 
scored  the  item.  Ranges  for  ihterreader  reliability  figures  for  mathematics  are  reported  in  Table  7-3. 
Average  reliabilities,  given  in  terms  of  percentage  exact  agreement,  for  each  booklet  type  (spiral, 
estimation,  theme,  and  advanced)  are  reported  in  Table  7-5.  This  reliability  information  was  also  used  by 
the  table  leader  in  monitoring  the  capabilities  of  all  readers  and  the  uniformity  of  scoring  across  readers. 
When  scoring  was  completed  for  an  item,  a hard  copy  of  the  report  was  printed  for  analysis  by  ETS 
project  staff. 


7.3.4.4  Short-Term  Trend 

The  1996  main  assessment  of  mathematics  included  a number  of  items  that  had  previously  been 
used  in  the  1990  and  1992  assessments.  A list  of  these  items  is  included  in  the  tables  in  Appendix  J.  For 
these  items,  the  trainers  used  the  same  scoring  guides  and  training  sets  as  in  1992.  Also,  for  those  items 
that  originated  in  the  1992  assessment,  750  responses  from  1992  were  scanned  into  the  scoring  system 
and  rescored.  For  those  items  that  originated  in  the  1990  assessment,  the  sample  included  750  responses 
each  from  1990  and  1992.  Table  leaders  used  the  management  tools  to  distribute  the  rescore  responses 
out  among  the  current-year  material.  Because  the  rescore  responses  could  not  be  seeded  into  the  current 
materials,  the  table  leaders  divided  the  rescore  material  into  thirds,  inserting  about  25  responses  per 
scorer  at  various  intervals  during  scoring.  Since  the  development  group  had  loaded  the  original  scores 


from  the  previous  years,  the  system  was  able  to  give  real-time  comparisons  of  scores.  Table  leaders, 
trainers,  and  the  NCS  and  ETS  subject  area  specialists  monitored  the  interreader  agreement  rates  and  t- 
tests  across  years  for  all  items  used  in  the  short-term  trend. 


7.3.4.S  Paper  Scoring 

Some  mathematics  items  could  not  be  scored  on  the  image  system  because,of  printing  problems 
or  difficulties  with  overlay  templates.  When  scoring  these  items,  professional  readers  coded  any  response 
that  could  not  be  scored  by  viewing  the  image  with  a designated  code  such  as  “8”  or  “pull.”  The 
development  group  created  a list  of  all  responses  coded  for  pulling,  and  the  project  coordinator  led  the 
effort  to  pull  these  booklets  from  the  warehouse.  The  booklets  were  then  transported  to  the  PSC,  where 
professional  readers  scored  them  using  the  same  scoring  guides,  but  marking  their  scores  on  scannable 
scoring  sheets.  Clerical  support  staff  then  scanned  the  sheets  and  uploaded  the  file  to  the  mainframe, 
where  the  development  team  merged  the  data  with  the  image  scoring  data. 

As  soon  as  the  last  item  on  any  score  sheet  was  completed,  the  score  sheets  were  collected  and 
taken  to  a central  clerical  support  area  to  be  scanned  on  the  NCS  paper-based  scoring  system  using 
OpScan  5 scanners.  As  each  sheet  was  processed,  the  scanning  system  compared  the  incoming  data  with 
tables  to  ensure  that  all  responses  were  scored  with  one  and  only  one  valid  score,  and  that  only  raters 
who  were  qualified  to  score  an  item  were  allowed  to  score  it.  Any  discrepancies  (e.g.,  no  score  assigned, 
double  gridding,  out-of-range  scores,  or  invalid  scorer  ID  numbers)  were  flagged  and  resolved  before  the 
data  from  that  sheet  were  accepted  into  the  scoring  system.  Interreader  agreement  reports  were  generated 
twice  a day. 

All  the  scoring  data  were  stored  on  personal  computers  at  NCS  after  all  the  responses  for  a 
subject  area  had  been  scanned.  Upon  completion  of  scoring,  the  scanner  operator  ran  a query  which 
compared  the  sheets  scanned  with  a table  of  records  in  the  system  to  make  sure  that  all  score  sheets  were 
accounted  for.  Once  all  edits  were  corrected,  the  personal  computer  file  was  renamed  and  put  into  an 
export  file,  which  automatically  created  the  mainfiame  file.  This  file  was  then  uploaded  to  the  mainframe 
to  be  merged  with  the  mainframe  student  files. 


7.3.4.6  Large  Print 

To  accommodate  students  with  visual  impairment,  field  administrators  had  Braille  and  large  print 
versions  of  designated  booklets  available  to  sampled  students  who  qualified  for  them.  The  scoring  center 
received  no  Braille  material  for  scoring.  Two  large  print  booklets  were  received  back  for  scoring.  One  of 
the  large  print  booklets  was  completely  blank.  The  NCS  performance  assessment  specialist  in  charge  of 
mathematics  scored  all  the  items  in  the  other  booklet  and  gave  the  scores  to  the  development  team  to 
enter  into  the  database. 


7.3.4.7  Bilingual  Scoring 

Some  students  who  participated  in  the  main  fourth-  and  eighth-grade  mathematics  assessments 
received  bilingual  booklets.  All  students  who  used  bilingual  booklets  received  the  equivalent  of  Booklet 
121  for  their  grade.  The  same  blocks  with  the  same  items  appeared  in  the  bilingual  booklets  as  in  the 
regular  Booklet  121,  with  the  exception  that  the  bilingual  booklets  contained  both  Spanish  and  English 
versions  of  each  item.  The  items  appeared  in  Spanish  on  the  left-hand  pages  and  in  English  on  the  right- 


hand  pages.  The  instructions  told  students  to  answer  according  to  their  preference.  Altogether,  the  PSC 
scored  91  bilingual  booklets  at  grade  4 and  36  at  grade  8. 

Because  of  the  small  number  of  booklets  involved,  the  PSC  selected  a team  of  four  readers  to 
score  all  of  the  booklets  on  paper.  The  team  consisted  of  two  males  and  two  females.  Two  of  the  scorers 
were  born  in  Chile,  South  America,  and  raised  partly  in  Chile  and  partly  in  the  United  States.  Both  of 
them  have  traveled  and  recently  lived  extended  periods  of  time  in  South  America,  while  making  the 
United  States  their  permanent  home.  The  third  scorer  comes  from  a Panamanian- American  background. 
Born  and  raised  in  a bilingual  family,  he  holds  a bachelor’s  degree  in  Spanish,  has  lived  extended  periods 
of  time  in  Panama  and  Mexico,  and  has  experience:  teaching  university-level  Spanish  as  well  as  working 
several  years  in  a business  position  that  required  daily  telephone  and  correspondence  contact  with 
Spanish-speaking  clientele,  mainly  from  Puerto  Rico.  The  fourth  scorer,  while  not  a native  speaker  of 
Spanish,  holds  a master’s  degree  in  Spanish,  has  traveled  and  studied  extensively  in  Mexico,  the 
Caribbean  and  South  America,  coordinated  an  adult  education  program  for  Spanish-speaking  immigrants 
from  Mexican  and  Central  American  backgrounds  for  three  years,  taught  university-level  Spanish  for 
eight  years,  and  worked  in  customer  relations  positions  for  four  years  dealing  with  Spanish-speaking 
clientele,  mainly  from  Puerto  Rico. 

The  NCS  mathematics  specialist  trained. all  four  readers  on  the  fourth-grade  items  on  Thursday, 
May  2,  1996,  and  the  team  scored  all  of  the  fourthTgrade  booklets  the  same  day.  On  Friday,  May  3,  the 
team  learned  the  eighth-grade  items,  many  of  which  overlapped  with  the  fourth-grade  items,  and  scored 
all  of  the  eighth-grade  booklets  on  that  day;  Twenty-five  percent  of  the  booklets  were  scored  by  a second 
reader  to  measure  interreader  agreement. 

The  team  applied  the  same  scoring  guides  that  were  used  for  regular  scoring.  However,  two 
scores  were  assigned  for  each  item,  one  for  the  Spanish  side  and  one  for  the  English  side.  Since  most 
students  answered  relatively  consistently  on  either  one  side  or  the  other,  most  booklets  received  on-task 
scores  for  one  language  and  blanks  for  the  other  language.  This  procedure  will  allow  analysts  to  separate 
the  data  of  those  students  who  answered  in  English  from  those  who  answered  in  Spanish.  Several 
students  wrote  their  answers  in  English  on  the  side  of  the  page  where  the  item  was  written  in  Spanish.  In 
these  cases,  the  scorers  coded  the  score  as  a Spanish  answer  since  that  is  where  the  student  read  the  item 
and  wrote  the  answer.  During  the  course  of  scoring,  the  team  noted  that  over  half  of  the  fourth-grade 
booklets  came  from  the  same  school,  and  not  a single  student  from  that  school  answered  any  questions  in 
Section  3 of  the  booklet. 

The  same  scanning  procedures  were  performed  as  outlined  in  Section  7. 3.4.5. 

7.3.5  Main  NAEP  Science 

The  science  portion  of  the  1996  NAEP  included  a total  of  374  discrete  constructed-response 
items  (see  Table  7-1).  It  was  scored  over  three  segments  and  two  shifts  (see  Table  7-2).  Many  kinds  of 
constructed-response  items  were  utilized  in  the  assessment  to  measure  different  elements  of  students’ 
conceptual  understanding  of  scientific  material  as  well  as  their  practical  reasoning  ability.  The  items 
scored  included  short-answer  constructed  responses  and  extended  constructed  responses.  Each 
constructed-response  item  had  a unique  scoring  guide  that  identified  the  range  of  possible  scores  for  the 
item  and  defined  the  criteria  to  be  used  in  evaluating  student  responses. 
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During  the  course  of  the  project,  each  team  scored  short  constructed-response  items  using  a scale 
that  allowed  for  partial  credit  as  follows: 

1 = incorrect  response 

2 = partial  understanding 

3 = correct  response 

The  readers  scored  extended  constructed-response  items  on  a scale  of  “1”  to  “4”  as  follows: 

1 = incorrect  response 

2 = minimal  understanding 

3 = satisfactory  level  of  comprehension 

4 = correct  response 


7.3.5.1  Training 

The  training  on  each  item  was  conducted  by  science  specialists  from  ETS  and  NCS.  The  first  teams 
began  training  on  March  18,  1996.  Other  teams  were  phased  in  throughout  the  project.  Hands-on  items 
were  scored  a block  at  a time  with  a unique  scoring  guide  for  each  item  because  of  the  related  nature  of  the 
items.  The  rest  of  the  assessment  was  scored  item-by-item  so  that  each  reader  worked  on  only  one  set  of 
rubrics  at  a time.  After  scoring  all  available  responses,  a team  would  then  proceed  with  training  and  scoring 
the  next  item.  Scoring  was  completed  on  June  7,  1996.  Table  7-2  gives  detailed  information  on  the  dates  of 
scoring  and  the  number  of  readers  and  table  leaders. 

Training  involved  explaining  the  item  and  its  scoring  guide  to  the  team  and  discussing  responses 
that  represented  the  various  score  points  in  the  guide.  Typically,  two  or  three  anchor  responses  were  chosen 
for  each  score  point.  During  this  stage,  readers  and  the  table  leader  kept  notes  of  scoring  decisions.  The 
table  leader  was  then  responsible  for  compiling  those  notes  and  ensuring  that  all  readers  were  in  alignment. 
When  review  of  the  anchor  packet  was  completed,  the  readers  scored  and  discussed  10  to  20  pre-scored 
“practice  papers”  that  represented  the  entire  range  of  score  points  the  item  could  receive.  After  the  trainer 
and  table  leader  determined  that  the  team  had  reached  consensus,  the  table  leader  then  released  work  on  the 
image-scoring  system  to  the  readers.  The  readers  would  initially  take  turns  reading  their  first  “live” 
responses  to  the  team  or  work  in  pairs  as  a final  check  before  beginning  work  individually.  Once  the 
practice  session  was  completed,  the  formal  scoring  process  began. 


7.3.S.2  Scoring 

All  scoring  for  science  was  conducted  via  the  image-based  scoring  system.  During  scoring,  the 
team  leaders  continued  to  compile  notes  on  scoring  decisions  for  the  readers’  reference  and  guidance. 
Additionally,  table  leaders  closely  monitored  interreader  reliability  using  both  team  and  individual  statistics 
as  a reference.  Consistently  throughout  the  scoring  of  each  item,  the  table  leaders  also  performed 
backreading  duties  in  which  they  reviewed  a sample  of  the  responses  scored  by  each  reader  on  the  team. 
Lead  scorers  selected  for  their  experience  and  accuracy  in  scoring  assisted  the  table  leaders  in  backreading. 
The  table  leaders  and  performance  assessment  specialist  continuously  monitored  the  progress  of  each  team 
and  noted  all  scoring-related  decisions  to  ensure  that  training  and  scoring  progressed  smoothly  and  in  a 
timely  manner. 
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Table  7-6 

Science  Constructed  Responses  Scored 

Type*  Assessment  Proportions 

Grade  Data Regular  Hands-On  Advanced  Grand  Total  1996  National  1996  State 
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One  advantage  of  utilizing  an  image-based  scoring  system  is  the  ability  to  construct  reader  aids  to 
simplify  scoring,  thus  increasing  reader  reliability.  Prior  to  the  start  of  the  project,  the  ETS  subject  area 
specialist  and  the  NCS  performance  assessment  specialist  identified  several  items  for  the  construction  of 
overlays.  Overlays  serve  as  templates  to  define  boundaries  in  which  correct  responses  must  be  located  or 
allow  the  placement  of  correct  answers  directly  on  the  displayed  image,  and  are  displayed  along  with  the 
student  response.  A schematic  representation  of  each  overlay  was  included  with  the  scoring  guide  and 
sample  papers  for  these  items  to  familiarize  readers  with  the  use  of  the  scoring  aids  during  training. 

General  information  on  the  number  of  constructed  responses  scored  can  be  found  in  Table  6-1  in 
Chapter  6.  Table  7-6  gives  more  detailed  information  by  grade  and  booklet  type  (spiral  and  advanced).  The 
codes  that  were  used  for  unscorable  science  items  were: 

0 = Blank  or  random  marks 

8 = Completely  cross-out  or  erased 

9 = “I  don’t  know,”  refusal,  off-task,  illegible  or  language  other  than  English  that  could  not 

be  translated 


7.3.S.3  Reliability 

A minimum  of  25  percent  of  the  science  responses  for  items  involved  only  in  the  national  sample  and  6 
percent  of  the  responses  for  items  involved  in  the  state  samples  were  scored  by  a second  reader  to  obtain 
statistics  on  interreader  reliability.  Ranges  for  interreader  reliability  for  science  are  reported  in  Table  7-3. 
Average  reliabilities,  given  in  terms  of  percentage  exact  agreement,  for  each  booklet  type  (spiral  and 
advanced)  are  reported  in  Table  7-6.  This  reliability  information  was  also  used  by  the  team  leaders  to  monitor 
the  capabilities  of  all  readers  and  maintain  uniformity  of  scoring  across  readers.  Reliability  reports  could  be 
generated  on  demand  by  the  table  leader,  scoring  specialist,  or  performance  assessment  specialist  when 
needed,  and  they  were  displayed  at  a computer  workstation.  In  addition  to  the  immediate  feedback  provided 
by  the  on-line  reliability  reports,  each  table  leader  could  also  review  the  actual  responses  scored  by  a reader 
by  using  the  backreading  tool.  In  this  way,  the  table  leader  monitored  each  reader  carefully  and  corrected 
difficulties  in  scoring  almost  immediately  with  a high  degree  of  efficiency. 
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Chapter  8 


CREATION  OF  THE  DATABASE,  QUALITY  CONTROL  OF 
DATA  ENTRY,  AND  CREATION  OF  THE  DATABASE  PRODUCTS^ 


The  data  processing,  scoring  and  editing  procedures  described  in  Chapter  6 resulted  in  the 
generation  of  disk  and  tape  files  containing  various  data  for  students  (assessed  and  excluded),  teachers, 
schools,  and  SD/LEP  (students  with  disabilities/students  with  limited  English  proficiency)  information. 
The  weighting  procedures  described  in  Chapter  10  resulted  in  the  generation  of  data  files  that  included 
the  sampling  weights  required  to  make  valid  statistical  inferences  about  the  population  from  which  the 
1996  fourth-,  eighth-  and  twelfth-grade  NAEP  samples  were  drawn.  These  files  were  merged  into  a 
comprehensive,  integrated  database.  The  creation  of  the  database  is  described  in  Section  8.2.1. 

Section  8.2.2  describes  a central  repository  or  master  catalog  of  this  information.  The  master 
catalog  is  accessible  by  all  analysis  and  reporting  programs  and  provides  correct  parameters  for 
processing  the  data  fields  and  consistent  labeling  for  identifying  the  results  of  the  analyses. 

To  evaluate  the  effectiveness  of  the  quality  control  of  the  data  entry  process,  the  corresponding 
portion  of  the  final  integrated  database  was  verified  in  detail  against  a sample  of  the  original  instruments 
received  from  the  field.  The  results  of  this  procedure  are  given  in  Section  8.3. 

The  integrated  database  was  the  source  for  the  creation  of  the  NAEP  item  information  database 
and  the  NAEP  secondary-use  data  files.  These  are  described  in  Section  8.4. 


8.2  CREATION  OF  THE  DATABASE 
8.2.1  Merging  Files 

The  data  processing  conducted  by  National  Computer  Systems  (NCS)  resulted  in  the  transmittal 
to  ETS  of  four  data  files  for  each  of  fourth,  eighth  and  twelfth  grade:  one  for  the  student  background  and 
item  response  data  and  one  file  for  each  of  the  three  questionnaires  (Teacher  Questionnaire,  School 
Characteristics  and  Policies  Questionnaire,  and  SD/LEP  Questionnaire).  The  sampling  weights,  derived 
by  Westat,  Inc.,  comprised  additional  files  for  each  grade — two  sets  for  assessed  students,  two  sets  for 
excluded  students  and  for  schools  four  sets  at  grade  4,  five  at  grade  8,  and  six  at  grade  12.  (See  Chapter 
10  for  a discussion  of  the  sampling  weights.)  These  files  at  each  grade  were  the  foundation  for  the 


* John  J.  Ferris  was  responsible  for  the  evaluation  of  the  quality  of  the  database  and  the  data  entry  process;  Katharine  E.  Pashley 
was  responsible  for  database  generation  under  the  supervision  of  David  S.  Freund;  Alfred  M.  Rogers  created  the  secondary-use 


John  J.  Ferris,  Katharine  E.  Pashley, 
David  S.  Freund,  and  Alfred  M.  Rogers 
Educational  Testing  Service 


8.1  INTRODUCTION 


data  files. 
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analysis  of  the  1996  NAEP  data.  Before  data  analyses  could  be  performed,  these  data  files  had  to  be 
integrated  into  a coherent  and  comprehensive  database. 

The  database  ultimately  comprised  four  files  per. cohort;  three  student  files  (mathematics, 
science,  and  long-term  trend)  and  a single  school  file.  The  student  files  were  separated  by  subject  area  to 
improve  maintenance  and  efficiency  of  the  databases  and  data  analyses.  Each  record  on  the  student  file 
contained  a student’s  responses  to  the  particular  assessment  booklet  the  student  was  administered  (in  the 
case  of  excluded  students,  a booklet  was  assigned  but  the  student  response  fields  contain  a special  code 
indicating  no  response),  and  the  information  from  the  questionnaire  that  the  student’s  teacher  completed. 
Additionally,  for  a student  (assessed  or  excluded)  who  was  identified  as  a student  with  a disability  (SD) 
or  of  limited  English  proficiency  (LEP),  the  data  from  the  SD/LEP  Questionnaire  are  included.  This 
questionnaire  is  filled  out  for  all  students  identified  as  SD  and/or  LEP,  both  assessed  and  excluded.  (See 
Chapter  2 for  information  regarding  assessment  instruments.)  Also  added  to  the  student  files  were 
variables  with  school-level  information  supplied  by  Quality  Education  Department,  Inc.  (QED)  including 
demographic  information  about  schools  such  as  race/ethnicity  percentages.  Since  the  teacher  data  are  not 
from  a representative  sample  of  teachers  and  since  the  focus  of  NAEP  is  to  report  student  level  results, 
the  teacher  response  data  were  added  to  the  student  records.  The  school  data  were  on  separate  files  that 
could  be  analyzed  on  their  own  and  could  also  be  linked  to  the  student  files  through  the  unique  school  ID 
code. 


The  creation  of  the  student  data  files  for  fourth,  eighth,  and  twelfth  grade  began  with  the 
reorganization  of  the  data  files  received  from  NCS.  This  involved  two  major  tasks; 

1.  the  files  were  restructured,  eliminating  unused  (blank)  areas  to  reduce  the  size  of  the 
files;  and 

2.  in  cases  where  students  had  chosen  not  to  respond. to  an  item,  the  missing  responses 
were  recoded  as  either  “omit”  or  “not  reached,”  as  discussed  in  Part  II  of  this  report. 

Next,  the  student  response  data  were  merged  with  the  student  weights  files.  The  resulting  file  was  then 
merged  with  the  SD/LEP  and  teacher  data.  In  all  merging  steps,  the  10-digit  booklet  ED  (the  three-digit 
booklet  number  common  to  every  booklet  with  the  same  block  of  items,  a six-digit  serial  number  unique 
to  the  booklet  a student  was  given  and  a single  check  digit,  distinguishing  bilingual  booklets)  was  used  as 
the  matching  criterion. 

The  school  file  for  each  grade  was  created  by  merging  the  School  Characteristics  and  Policies 
Questionnaire  file  with  the  file  of  school  weights  and  school  variables,  supplied  by  Westat.  The  primary 
sampling  unit  (PSU)  and  school  codes  were  used  as  the  matching  criteria.  Since  some  schools  did  not 
return  a questionnaire,  some  of  the  records  in  the  school  file  contained  only  school-identifying 
information  and  sampling  weight  information. 

When  the  student  and  school  files  for  each  grade  had  been  created,  the  database  was  ready  for 
analysis.  In  addition,  whenever  new  data  values,  such  as  composite  background  variables  or  plausible 
values,  were  derived,  they  were  added  to  the  appropriate  database  files  using  the  same  matching 
procedures  described  above. 

For  archival  purposes  and  to  provide  data  for  outside  users,  restricted-use  data  files  and 
codebooks  for  each  jurisdiction  were  generated  from  this  database.  The  restricted-use  data  files  contain 
all  responses  and  response-related  data  from  the  assessment,  including  responses  from  the  student 
booklets.  Teacher  Questionnaires,  and  School  Characteristics  and  Policies  Questionnaires,  scale  scores, 
sampling  weights,  and  variables  used  to  compute  standard  errors. 
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8.2.2  Creating  the  Master  Catalog 


A critical  part  of  any  database  is  its  processing  control  and  descriptive  information.  Having  a 
central  repository  for  this  information,  which  may  be  accessed  by  all  analysis  and  reporting  programs, 
will  provide  correct  parameters  for  processing  the  data  fields  and  consistent  labeling  for  identifying  the 
results  of  the  analyses.  The  NAEP  master  catalog  file  was  designed  and  constructed  to  serve  these 
purposes  for  the  NAEP  database. 

Each  record  of  the  master  catalog  contains  the  processing,  labeling,  classification,  and  location 
information  for  each  data  field  in  the  NAEP  database.  The  control  parameters  are  used  by  the  access 
routines  in  the  analysis  programs  to  define  the  manner  in  which  the  data  values  are  to  be  transformed  and 
processed. 

Each  data  field  has  a 50-character  label  in  the  master  catalog  describing  the  contents  of  the  field 
and,  where  applicable,  the  source  of  the  field.  The  data  fields  with  discrete  or  categorical  response  values 
(e.g.^  multiple-choice  items  and  professionally  scored  items,  but  not  weight  fields)  have  additional  label 
fields  in  the  catalog  containing  8-  and  20-character  labels  for  those  response  values.  These  short  labels 
can  be  used  for  reporting  purposes  as  a concise  description  of  the  responses  for  the  cognitive  items. 

The  classification  area  of  the  master  catalog  record  contains  distinct  fields  corresponding  to 
predefined  classification  categories  (e.g.,  mathematics  content  and  process  areas)  for  the  data  fields.  For 
a particular  classification  field,  a nonblank  value  indicates  the  code  of  the  subcategory  within  the 
classification  category  for  the  data  field.  This  classification  area  permits  the  grouping  of  identically 
classified  items  or  data  fields  by  performing  a selection  process  on  one  or  more  classification  fields  in  the 
master  catalog. 

According  to  NAEP  design,  it  is  possible  for  item  data  fields  to  appear  in  more  than  one 
student  sample  and  in  more  than  one  block  within  each  sample.  The  location  fields  of  the  catalog  record 
contain  age,  block  and,  where  applicable,  the  sequence  within  the  block  for  each  appearance  of  the  data 
field. 

The  master  catalog  file  was  constructed  concurrently  with  the  collection  and  transcription  of  the 
State  Assessment  data  so  that  it  would  be  ready  for  use  by  analysis  programs  when  the  database  was 
created.  As  new  data  fields  were  derived  and  added  to  the  database,  their  corresponding  descriptive  and 
control  information  were  entered  into  the  master  catalog.  The  machine-readable  catalog  files  are 
available  as  part  of  the  secondary-use  data  files  package  for  use  in  analyzing  the  data  with  programming 
languages  such  as  SAS  and  SPSS-X  (see  the  NAEP  1996  Secondary-Use  Data  Files  User  Guide,  Rogers, 
Kline,  & Schoeps,  1999). 


8.3  QUALITY  CONTROL  OF  NAEP  DATA  ENTRY  FOR  1996 

This  section  describes  the  evaluation  of  the  data  entry  process  for  the  1996  national  assessment. 
As  in  past  years,  the  NAEP  database  was  found  to  be  more  than  accurate  enough  to  support  the  analyses 
that  were  done.  Overall,  the  observed  error  rates  were  comparable  to  those  of  past  assessments,  with  the 
possible  exception  of  the  Teacher  Questionnaire  data  (see  discussion  below);  they  ranged  from  three 
errors  per  10,000  responses  for  the  SD/LEP  Student  Questionnaire  data  to  33  errors  per  10,000  responses 
for  the  Teacher  Questionnaire  data. 
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The  purpose  of  the  analysis  reported  in  this  sction  is  to  assess  the  quality  of  the  data  resulting 
from  the  complete  data  entry  system,  beginning  with  the  actual  instruments  collected  in  the  field  and 
ending  with  the  final  machine-readable  database  used  in  the  analyses.  The  process  involved  the  selection 
of  instruments  at  random  from  among  those  returned  from  the  field  and  the  comparison  of  these 
instruments,  character  by  character,  with  their  representations  in  the  final  database.  In  this  way,  we  were 
able  to  measure  the  error  rates  in  the  data  as  well  as  the  success  of  the  data  entry  system. 

Of  course  the  observed  error  rate  cannot  be  taken  at  face  value.  For  example,  the  sample  of 
teacher  questionnaires  that  happened  to  be  selected  for  close  inspection  contained  22  errors  out  of  a total 
of  6,741  characters.  To  conclude  that  the  entire  teacher  questionnaire  database  has  an  error  rate  of 
22/6741,  or  .0033,  would  be  too  optimistic;  we  may  simply  have  been  lucky  (or  unlucky)  with  this 
particular  random  sample.  What  is  needed  is  an  indication  of  how  bad  the  true  error  rate  might  be,  given 
what  we  observed.  Such  an  indication  is  provided  by  confidence  limits.  Confidence  limits  indicate  how 
likely  it  is  that  a value  falls  inside  a specified  range  in  a specified  context  or  distribution.  In  our  analysis, 
the  specified  range  is  an  error  rate  between  zero  and  some  maximum  value  beyond  which  we  are 
confident  at  a specified  level  (traditionally  99.8%)  that  the  true  error  rate  does  not  lie;  the  specified 
context  or  distribution  turns  out  to  be  the  cumulative  binomial  probability  distribution.  An  example 
should  demonstrate  this  technique: 

Let  us  say  that  1,000  booklets  were  processed,  each  with  100  characters  of  data 
transcribed  for  a total  of  100,000  characters.  Let  us  say  further  that  five  of  these 
characters  were  discovered  to  be  in  error  in  a random  sample  of  50  booklets  that  were 
completely  checked;  in  other  words,  five  errors  were  found  in  a sample  of  5,000 
characters.  The  following  expression  may  be  used  to  establish  the  probability  that  the 
true  error  rate  is  .0025  or  less,  rather  than  the  single-value  estimate  of  the  observed  rate 
of  one  in  a thousand  (.001): 
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This  is  the  sum  of  the  probability  of  finding  five  errors  plus  the  probability  of  finding 
four  errors  plus.  . . etc.  . . plus  the  probability  of  finding  zero  errors  in  a sample  of  5,000 
with  a true  error  rate  of  .0025;  that  is,  the  probability  of  finding  five  or  fewer  errors  by 
chance  when  the  true  error  rate  is  .0025.  Notice  that  we  did  not  use  the  size  of  the 
database  in  this  expression.  Actually,  the  assumption  here  is  that  our  sample  of  5,000 
was  drawn  from  a database  that  is  infinite.  The  smaller  the  actual  database  is,  the  more 
confidence  we  can  have  in  the  observed  error  rate;  for  example,  had  there  been  only 
5,000  in  the  total  database,  our  sample  would  have  included  all  the  data,  and  the 
observed  error  rate  would  have  been  the  true  error  rate.  The  result  of  the  above 
computation  allows  us  to  say,  conservatively,  that  .0025  is  an  upper  limit  on  the  true 
error  rate  with  98.53  percent  (i.e.,  1 - .0147)  confidence;  that  is,  we  can  be  quite  sure  that 
our  true  error  rate  is  no  larger  than  .0025. 

Virtually  all  of  the  data  collected  for  this  assessment  were  machine-scanned.  The  only  exception 
was  a set  of  six  booklets  used  for  the  long-term  trend  reading  and  writing  assessments;  the  format 
these  booklets  was  kept  the  same  for  comparability  with  earlier  assessments,  so  these  booklets  had  to  be 
key -entered.  As  it  happened,  no  errors  at  all  were  found  in  the  sample  of  key -entered  booklets  selected 
for  quality  control.  z 
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In  the  1994  and  1996  assessments,  the  selection  of  booklets  for  this  comparison  took  place  at  the 
point  of  first  entry  into  the  recording  process  for  data  from  the  field.  In  earlier  assessments,  this  selection 
took  place  only  after  data  had  reached  the  final  database,  in  order  to  assure  that  only  relevant  booklets 
were  involved  in  the  quality  control  evaluation.  The  new  selection  process  involves  the  risk  that  booklets 
will  be  selected  that  ultimately  will  not  appear  in  the  final  database,  however,  as  in  1994,  sufficient 
numbers  of  booklets  were  in  fact  selected. 

The  individual  instruments  are  briefly  discussed  in  the  following  sections  and  a summary  table 
(Table  8-1)  gives  the  upper  99.8  percent  confidence  limit  for  the  error  rate  for  each  of  the  instruments  as 
well  as  the  sampling  information.  The  99.8  percent  confidence  limit,  and  the  selection  rates  noted,  were 
chosen  to  make  these  results  comparable  to  those  of  previous  administrations  when  the  same  parameters 
were  used. 


Table  8-1 

Summary  of  Quality  Control  Error  Analysis  for  NAEP  1996  Data  Entry 


Main 

Assessment 

Student 

Booklets 

Long-Term 
Trend 
Assessment 
Student  Booklets 

SD/LEP 

Student 

Questionnaires 

Teacher 

Questionnaires 

School 

Characteristics 
and  Policies 
Questionnaires 

Selection  Rate 

1/392 

1/372 

1/112 

1/120 

1/57 
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8.3.1  Student  Data 

Data  from  about  94,000  students  were  processed  across  all  samples  in  this  assessment.  Across  all 
the  student  data,  roughly  one  booklet  in  392  was  selected  for  close  examination,  which  is  comparable  to 
the  one  in  400  target  selection  rate  used  in  past  assessments.  The  student  data  error  rates  were 
consistently  low  in  all  subject  areas  and  across  all  three  grades.  The  overall  quality  of  the  data  was  very 
high. 


Data  from  some  30,000  additional  students  were  also  used  in  the  study  of  long-term  trends. 
These  data  showed  the  same  consistently  low  error  rates,  as  indicated  in  the  accompanying  table. 


ERIC 


195 


171 


8.3.2  SD/LEP  Student  Questionnaire  Data 


In  this  assessment,  13,098  SD/LEP  questionnaires  were  scanned.  The  quality  control  sampling 
rate  was  one  in  112,  a somewhat  higher  rate  than  that  used  in  previous  assessments.  The  data  showed  a 
somewhat  lower  error  rate  than  in  previous  assessments — comparable  to  that  for  the  student  data.  The 
few  problems  encountered  involved  the  scanner’s  mistaking  an  erasure  for  a genuine  response  or  failing 
to  identify  a multiple  response  as  such. 


8.3.3  Teacher  Questionnaire  Data 

In  this  assessment,  4,585  teacher  questionnaires  were  collected  and  scanned.  About  one 
percent  of  these  questionnaires  was  sampled  for  the  quality  control  procedure.  The  error  rates  for  these 
questionnaires  were  higher  than  those  of  any  other  category  of  data,  and  also  higher  than  those  found  for 
teachers  in  past  assessments.  There  was  some  evidence  that  the  questionnaire  layout  was  confusing  to  at 
least  some  of  the  teachers  in  the  part  of  the  questionnaire  that  was  intended  to  collect  class  period 
information.  While  the  majority  of  teachers  did  not  have  difficulty  with  this,  consideration  is  being  given 
to  a possible  revision  of  the  questionnaire,  since  a number  of  teachers  also  had  similar  problems  in  the 
NAEP  State  Assessment. 


8.3.4  School  Characteristics  and  Policies  Questionnaire  Data 

In  this  assessment,  2,404  School  Characteristics  and  Policies  Questionnaires  were  collected. 

They  were  sampled  at  a rate  of  about  1 in  57.  Fifteen  scanning  errors  were  found  in  these  questionnaires, 
which  included  both  regular  and  long-term  trend  schools.  Most  of  these  errors  came  from  a single  booklet 
that  had  been  filled  out  in  pen — something  that  frequently  gives  a scanner  trouble. 


8.4  NAEP  DATABASE  PRODUCTS 

The  NAEP  database  described  to  this  point  serves  primarily  to  support  analysis  and  reporting 
activities  that  are  directly  related  to  the  NAEP  contract.  This  database  has  a singular  structure  and  access 
methodology  that  is  integrated  with  the  NAEP  analysis  and  reporting  programs.  One  of  the  directives  of 
the  NAEP  contract  is  to  provide  secondary  researchers  with  a nonproprietary  version  of  the  database  that 
is  portable  to  any  computer  system.  In  the  event  of  transfer  of  NAEP  to  another  client,  the  contract 
further  requires  ETS  to  provide  a full  copy  of  the  internal  database  in  a format  that  may  be  installed  on  a 
different  computer  system. 

In  fulfillment  of  these  requirements,  ETS  provides  two  sets  of  database  products:  the  item 
information  database  and  the  secondary-use  data  files.  The  contents,  format  and  usage  of  these  products 
are  documented  in  the  publications  listed  under  the  appropriate  sections  below. 


8.4.1  The  Item  Information  Database 


The  NAEP  item  information  database  contains  all  of  the  descriptive,  processing,  and  usage 
information  for  every  assessment  item  developed  and  used  for  NAEP  since  1970.  The  primary  unit  of  this 
database  is  the  item.  Each  NAEP  item  is  associated  with  different  levels  of  information,  including  usage 
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across  years  and  age  cohorts,  subject  area  classifications,  response  category  descriptors,  and  locations  of 
response  data  on  secondary-use  data  files. 

The  item  information  database  is  used  for  a variety  of  essential  NAEP  tasks;  providing  statistical 
information  to  aid  in  test  construction,  determining  the  usage  of  items  across  assessment  years  and  ages 
for  trend  and  “main”  analyses,  labeling  summary  analyses  and  reports,  and  organizing  items  by  subject 
area  classifications  for  scaling  analysis. 

The  creation,  structure,  and  use  of  the  NAEP  item  information  database  for  all  items  used  up  to 
and  including  the  1996  assessment  are  fully  documented  in  the  NAEP  publications  A Guide  to  the  NAEP 
Item  Information  Database  (Rogers,  Barone,  & Kline,  1996)  and  A Primer  for  the  NAEP  Item 
Information  Database  (Rogers,  Kline,  Barone,  Mychajlowycz,  & Forer,  1989). 

The  procedures  used  to  create  the  1996  version  of  the  item  information  database  are  the  same  as 
those  documented  in  the  guide.  The  version  of  the  guide  contains  the  subject  area  classification 
categories  for  the  cognitive  items. 


8.4.2  The  Secondary-Use  Data  Files 

The  secondary-use  data  files  are  designed  to  enable  any  researcher  with  an  interest  in  the  NAEP 
database  to  perform  secondary  analysis  on  the  same  data  as  those  used  at  ETS.  The  data,  documentation 
and  supporting  files  are  distributed  on  CD-ROM  media.  For  each  sample  in  the  assessment,  the  following 
files  are  provided;  the  response  data  file;  a printable  codebook  file;  a file  of  control  statements  that  will 
generate  an  SPSS  system  file;  a file  of  control  statements  that  will  generate  a SAS  system  file;  and  a 
machine-readable  catalog  file.  Each  codebook  file  is  in  portable  document  file  (PDF)  format,  which  may 
be  browsed,  excerpted  and  printed  using  the  Adobe  Acrobat  Reader  program  on  a variety  of  platforms. 
Each  machine-readable  catalog  file  contains  sufficient  control  and  descriptive  information  to  permit  the 
user  who  does  not  have  either  SAS  or  SPSS  to  set  up  and  perform  data  analysis. 

The  remainder  of  this  section  summarizes  the  procedures  used  in  generating  the  data  files  and 
related  materials. 


8.4.2.1  File  Definition 

The  design  of  the  1996  assessment  perpetuates  two  features  of  the  1990,  1992,  and  1994 
assessment  design;  the  focused-BIB  booklet  design  and  the  direct  matching  of  teacher  questionnaires  to 
student  assessment  instruments.  In  addition,  the  sample  of  students  who  were  excluded  from  the 
assessment  is  now  incorporated  into  the  appropriate  assessed  student  subject  area  sample. 

The  focused-BIB  design  within  the  main  assessment  isolates  the  primary  subject  areas  to  separate 
groups  of  booklets.  This  permits  the  division  of  the  main  sample  into  subject-specific  subsamples.  The 
data  files  generated  from  these  subsamples  need  only  contain  the  data  that  are  relevant  to  their 
corresponding  subject  areas  and  are  therefore  smaller  and  more  manageable  than  their  counterparts  in 
previous  assessments. 

According  to  the  design  of  the  1984,  1986,  and  1988  assessments,  only  a sample  of  the  teachers 
of  the  assessed  students  were  asked  to  fill  out  the  teacher  questionnaires.  The  large  size  of  the 
secondary-use  main  student  files  and  the  relatively  low  matching  rate  between  students  and  teachers 
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made  it  impractical  if  not  physically  prohibitive  to  produce  a complete  file  with  student  and  teacher 
information.  Both  the  1984  and  1986  secondary-use  data  packages  had  separate  teacher  data  files  which 
could  be  linked  to  the  student  data  files  for  analysis.  The  teacher  file  in  the  1988  secondary-use  data 
package  contained  not  only  the  teacher  response  data,  but  also  the  data  from  the  students  who  could  be 
matched  to  teacher  questionnaires.  This  type  of  file  was  more  appropriate  for  the  analysis  of  teacher  data 
because  it  defined  the  student  as  the  unit  of  observation. 

The  intent  of  the  1996  assessment  design  was  to  collect  data  from  mathematics  or  science 
teachers  of  the  main  assessment  students  at  specified  grade  levels  who  were  administered  mathematics  or 
science  booklets.  A portion  of  the  teacher  questionnaire  contained  questions  that  were  directly  related  to 
each  matched  student.  This  change  in  the  design  afforded  a very  high  matching  rate  between  student  and 
teacher  data.  Therefore,  for  those  subject  areas  in  each  grade  cohort  for  which  teacher  data  were 
collected,  the  teacher  responses  were  appended  to  each  student  record  in  the  secondary-use  data  files. 


S.4.2.2  Deflnition  of  the  Variables 


The  initial  step  in  the  variable  definition  process  was  the  generation  of  a LABELS  file  of 
descriptors  of  the  variables  for  each  data  file  to  be  created.  Each  record  in  a LABELS  file  contains,  for  a 
single  data  field,  the  variable  name,  a short  description  of  the  variable,  and  processing  control 
information  to  be  used  by  later  steps  in  the  data  generation  process.  This  file  could  be  edited  for  deletion 
of  variables,  modification  of  control  parameters,  or  reordering  of  the  variables  within  the  file.  The 
LABELS  file  is  an  intermediate  file  only;  it  is  not  included  on  the  released  data  files. 

The  variables  on  all  data  files  are  grouped  and  arranged  in  the  following  order:  identification 
information,  weights,  derived  variables,  proficiency  scores  (where  applicable),  and  response  data.  On  the 
student  data  files,  these  fields  are  followed  by  the  teacher  response  data  and  the  SD/LEP  student 
questionnaire  data,  where  applicable.  The  identification  information  is  taken  from  the  front  covers  of  the 
instruments.  The  weight  data  include  sample  descriptors,  selection  probabilities,  and  replicate  weights  for 
the  estimation  of  sampling  error.  The  derived  data  include  sample  descriptions  from  other  sources  and 
variables  that  are  derived  from  the  response  data  for  use  in  analysis  or  reporting. 


For  each  subject  area  of  the  main  assessment,  the  item  response  data  within  each  block  were  left 
in  their  order  of  presentation.  The  blocks,  however,  were  arranged  according  to  the  following  scheme: 
common  background,  subject-related  background,  the  cognitive  blocks  in  ascending  numerical  order,  and 
student  motivation.  The  responses  to  cognitive  blocks  that  were  not  present  in  a given  booklet  were  left 
blank,  signifying  a condition  of  “missing  by  design.” 

In  order  to  process  and  analyze  the  spiral  sample  data  effectively,  the  user  must  also  be  able  to 
determine,  from  a given  booklet  record,  which  blocks  of  item  response  data  were  present  and  their 
relative  order  in  the  instrument.  This  problem  was  remedied  by  the  creation  of  a set  of  control  variables, 
one  for  each  block,  which  indicated  not  only  the  presence  or  absence  of  the  block  but  its  order  in  the 
instrument.  These  control  variables  are  included  with  the  derived  variables. 


S.4.2.3  Data  Deflnition 


To  enable  the  data  files  to  be  processed  on  any  computer  system  using  any  procedural  or 
programming  language,  it  was  desirable  that  the  data  be  expressed  in  numeric  format.  This  was  possible, 
but  not  without  the  adoption  of  certain  conventions  for  reexpressing  the  data  values. 
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During  creation  of  the  NAEP  database,  the  responses  to  all  multiple-choice  items  were 
transcribed  and  stored  in  the  database  using  the  letter  codes  printed  in  the  instruments.  This  scheme 
afforded  the  advantage  of  saving  storage  space  for  items  with  10  or  more  response  options,  but  at  the 
expense  of  translating  these  codes  into  their  numeric  equivalents  for  analysis  purposes.  The  response  data 
fields  for  most  of  these  items  would  require  a simple  alphabetic-to-numeric  conversion.  However,  the 
data  fields  for  items  with  10  or  more  response  choices  would  require  “expansion”  before  the  conversion, 
since  the  numeric  value  would  require  two  column  positions.  One  of  the  processing  control  parameters 
on  the  LABELS  file  indicates  whether  or  not  the  data  field  is  to  be  expanded  before  conversion  and 
output. 

The  ETS  database  contained  special  codes  to  indicate  certain  response  conditions:  “I  don’t 
know”  responses,  multiple  responses,  omitted  responses,  not-reached  responses,  and  unresolvable 
responses,  which  include  out-of-range  responses  and  responses  that  were  missing  due  to  errors  in  printing 
or  processing.  The  scoring  guides  for  the  mathematics  and  science  constructed-response  items  included 
additional  special  codes  for  ratings  of  “illegible,”  “I  don’t  know,”  “off  task,  or  non-rateable  by  the 
scorers.  All  of  these  codes  had  to  be  reexpressed  in  a consistent  numeric  format. 

The  following  convention  was  adopted  and  used  in  the  designation  of  these  codes:  The 
“illegible”  response  codes  were  converted  to  5;  the  “off  task”  response  codes  were  converted  to  6;  the  “I 
don’t  know”  and  non-rateable  response  codes  were  converted  to  7;  the  “omitted”  response  codes  were 
converted  to  8;  the  “not  reached”  response  codes  were  converted  to  9;  and  the  multiple  response  codes 
were  converted  to  0;  and.  The  out-of-range  and  missing  responses  were  coded  as  blank  fields, 
corresponding  to  the  “missing  by  design”  designation. 

This  coding  scheme  created  conflicts  for  those  multiple-choice  items  that  had  seven  or  more 
valid  response  options  as  well  as  the  “I  don’t  know”  response  and  for  those  constructed-response  items 
whose  scoring  guide  had  five  or  more  categories.  These  data  fields  were  also  expanded  to  accommodate 
the  valid  response  values  and  the  special  codes.  In  these  cases,  the  special  codes  were  “extended”  to  fill 
the  output  data  field:  the  “I  don’t  know”  and  non-rateable  codes  were  extended  from  7 to  77,  omitted 
response  codes  from  8 to  88,  etc. 

Each  numeric  variable  on  the  secondary-use  files  was  classified  as  either  continuous  or  discrete. 
The  continuous  variables  include  the  weights,  proficiency  scores,  identification  codes,  and  item 
responses  where  counts  or  percentages  were  requested.  The  discrete  variables  include  those  items  for 
which  each  numeric  value  corresponds  to  a response  category.  The  designation  of  “discrete”  also 
includes  those  derived  variables  to  which  numeric  classification  categories  have  been  assigned.  The 
constructed-response  items  were  treated  as  a special  subset  of  the  discrete  variables  and  were  assigned  to 
a separate  category  to  facilitate  their  identification  in  the  documentation. 


8.4.2.4  Data  File  Catalogs 

The  CATALOG  file  is  created  by  the  GENCAT  program  from  the  LABELS  file  and  the  1996 
master  catalog  file.  Each  record  on  the  LABELS  file  generates  a CATALOG  record  by  first  retrieving  the 
master  catalog  record  corresponding  to  the  field  name.  The  master  catalog  record  contains  usage, 
classification,  and  response  code  information,  along  with  positional  information  from  the  LABELS  file, 
field  sequence  number,  output  column  position,  and  field  width.  Like  the  LABELS  file,  the  CATALOG 
file  is  an  intermediate  file  and  is  not  included  on  the  released  data  files. 
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The  information  for  the  response  codes,  also  referred  to  as  “foils,”  consists  of  the  valid  data 
values  for  the  discrete  numeric  fields,  and  a 20-character  description  of  each.  The  GENCAT  program 
uses  additional  control  information  from  the  LABELS  file  to  determine  if  extra  foils  should  be  generated 
and  saved  with  each  CATALOG  record.  The  first  flag  controls  generation  of  the  “I  don’t  know”  or  non- 
rateable  foil;  the  second  flag  regulates  omitted  or  not-reached  foil  generation;  and  the  third  flag  denotes 
the  possibility  of  multiple  responses  for  that  field  and  sets  up  an  appropriate  foil.  All  of  these  control 
parameters,  including  the  expansion  flag,  may  be  altered  in  the  LABELS  file  by  use  of  a text  editor,  in 
order  to  control  the  generation  of  data  or  descriptive  information  for  any  given  field. 

The  LABELS  file  supplies  control  information  for  many  of  the  subsequent  secondary-use  data 
processing  steps.  The  CATALOG  file  provides  detailed  information  for  those  and  other  steps. 


8.4.2.S  Data  File  Layouts 

The  data  file  layouts  were  the  first  user  product  to  be  generated  in  the  secondary-use  data  files 
process.  The  generation  program,  GENLYT,  used  a CATALOG  file  as  input  and  produced  a printable 
file.  The  LAYOUT  file  is  little  more  than  a formatted  listing  of  the  CATALOG  file. 

Each  line  of  the  LAYOUT  file  contains  the  following  information  for  a single  data  field: 
sequence  number,  field  name,  output  column  position,  field  width,  number  of  decimal  places,  data  type, 
value  range,  key  or  correct  response  value,  and  a short  description  of  the  field.  The  sequence  number  of 
each  field  is  implied  from  its  order  on  the  LABELS  file.  The  field  name  is  an  8-character  label  for  the 
field  that  is  to  be  used  consistently  by  all  secondary-use  data  files  materials  to  refer  to  that  field  on  that 
file.  The  output  column  position  is  the  relative  location  of  the  beginning  of  that  field  on  each  record  for 
that  file,  using  bytes  or  characters  as  the  unit  of  measure.  The  field  width  indicates  the  number  of 
columns  used  in  representing  the  data  values  for  a field.  If  the  field  contains  continuous  numeric  data,  the 
value  under  the  number  of  decimal  places  entry  indicates  how  many  places  to  shift  the  decimal  point 
before  processing  data  values. 

The  data  type  category  uses  five  codes  to  designate  the  nature  of  the  data  in  the  field:  Continuous 
numeric  data  are  coded  “C;”  discrete  numeric  data  are  coded  “D;”  constructed-response  item  data  are 
coded  OS;  if  the  item  was  dichotomized  for  scaling  and  “OE;”  if  it  was  scaled  under  a polytomous 
response  model.  Additionally,  the  discrete  numeric  fields  that  include  “I  don’t  know”  response  codes  are 
coded  DI.  If  the  field  type  is  discrete  numeric,  the  value  range  is  listed  as  the  minimum  and  maximum 
permitted  values  separated  by  a hyphen  to  indicate  range.  If  the  field  is  a response  to  a scorable  item,  the 
correct  option  value,  or  key,  is  printed;  if  the  field  is  an  assigned  score  that  was  scaled  as  a dichotomous 
item  using  cut  point  scoring,  the  range  of  correct  scores  is  printed.  Each  variable  is  further  identified  by  a 
50-character  descriptor. 


S.4.2.6  Data  Codebooks 

The  data  codebook  is  a printed  document  containing  complete  descriptive  information  for  each 
data  field.  Most  of  this  information  originates  from  the  CATALOG  file;  the  remaining  data  comes  from 
the  COUNTS  file  and  the  IRT  parameters  file. 

Each  data  field  receives  at  least  one  line  of  descriptive  information  in  the  codebook.  If  the  data 
type  is  continuous  numeric,  no  more  information  is  given.  If  the  variable  is  discrete  numeric,  the 
codebook  lists  the  foil  codes,  foil  labels,  and  frequencies  of  each  value  in  the  data  file.  Additionally,  if 
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the  field  represents  an  item  used  in  IRT  scaling,  the  codebook  lists  the  parameters  used  by  the  scaling 
program. 

Certain  blocks  of  cognitive  items  in  the  1996  assessment  that  are  to  be  used  again  in  later 
assessments  for  trend  comparisons  have  been  designated  as  nonreleased.  In  order  to  maintain  their 
confidentiality,  generic  labels  have  been  substituted  for  the  response  category  descriptions  of  these  items  . 
in  the  data  codebooks  and  the  secondary-use  files. 

The  frequency  counts  are  not  available  on  the  catalog  file,  but  must  be  generated  from  the  data. 
The  GENFREQ  program  creates  the  COUNTS  file  using  the  field  name  to  locate  the  variable  in  the 
database,  and  the  foil  values  to  validate  the  range  of  data  values  for  each  field.  This  program  also  serves 
as  a check  on  the  completeness  of  the  foils  in  the  CATALOG  file,  as  it  flags  any  data  values  not 
represented  by  a foil  value  and  label. 

The  IRT  parameter  file  is  linked  to  the  CATALOG  file  through  the  field  name.  Printing  of  the 
IRT  parameters  is  governed  by  a control  flag  in  the  classification  section  of  the  CATALOG  record.  If  an 
item  has  been  scaled  for  use  in  deriving  the  proficiency  estimates,  the  IRT  parameters  are  listed  to  the 
right  of  the  foil  values  and  labels,  and  the  score  value  for  each  response  code  is  printed  to  the  immediate 
right  of  the  corresponding  frequency. 

The  LAYOUT  and  CODEBOOK  files  are  written  by  their  respective  generation  programs  to 
print-image  disk  data  files.  Draft  copies  are  printed  and  distributed  for  review  before  the  production  copy 
is  generated.  The  production  copy  combines  the  LAYOUT  and  CODEBOOK  files  for  each  sample  in  a 
portable  document  file  (PDF)  format.  This  file  may  be  browsed,  excerpted  and  printed  using  the  Adobe 
Acrobat  Reader  program  on  a variety  of  platforms  and  operating  systems. 


8.4.2.7  Control  Statement  Files  for  Statistical  Packages 

An  additional  requirement  of  the  NAEP  cooperative  agreement  is  to  provide,  for  each 
secondary-use  data  file,  a file  of  control  statements  each  for  the  SAS  and  SPSS  statistical  systems  that 
will  convert  the  raw  data  file  into  the  system  data  file  for  that  package.  Two  separate  programs,  GENSAS 
and  GENSPX,  generate  these  control  files  using  the  CATALOG  file  as  input. 

Each  of  the  control  files  contains  separate  sections  for  variable  definition,  variable  labeling, 
missing  value  declaration,  value  labeling,  and  creation  of  scored  variables  from  the  cognitive  items.  The 
variable  definition  section  describes  the  locations  of  the  fields,  by  name,  in  the  file,  and,  if  applicable,  the 
number  of  decimal  places  or  type  of  data.  The  variable  label  identifies  each  field  with  a 50-character 
description.  The  missing  value  section  identifies  values  of  those  variables  that  are  to  be  treated  as 
missing  and  excluded  from  analyses.  The  value  labels  correspond  to  the  foils  in  the  CATALOG  file.  The 
code  values  and  their  descriptors  are  listed  for  each  discrete  numeric  variable.  The  scoring  section  is 
provided  to  permit  the  user  to  generate  item  score  variables  in  addition  to  the  item  response  variables. 

Each  of  the  code  generation  programs  combines  three  steps  into  one  complex  procedure.  As  each 
CATALOG  file  record  is  read,  it  is  broken  into  several  component  records  according  to  the  information 
to  be  used  in  each  of  the  resultant  sections.  These  record  fragments  are  tagged  with  the  field  sequence 
number  and  a section  sequence  code.  They  are  then  organized  by  section  code  and  sequence  number. 
Finally,  the  reorganized  information  is  output  in  a structured  format  dictated  by  the  syntax  of  the 
processing  language. 
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The  generation  of  the  system  files  accomplishes  the  testing  of  these  control  statement  files.  The 
system  files  are  saved  for  use  in  special  analyses  by  NAEP  staff.  These  control  statement  files  are 
included  on  the  distributed  data  files  to  permit  users  with  access  to  SAS  and/or  SPSS  to  create  their  own 
system  files. 


S.4.2.8  Machine-Readable  Catalog  Files 

For  those  NAEP  data  users  who  have  neither  SAS  nor  SPSS  capabilities,  yet  require  processing 
control  information  in  a computer-readable  format,  the  distribution  files  also  contain  machine-readable 
catalog  files.  Each  machine-readable  catalog  record  contains  processing  control  information,  IRT 
parameters,  and  foil  codes  and  labels. 


S.4.2.9  NAEP  Data  on  Disk 

The  complete  set  of  secondary-use  data  files  described  above  are  available  on  CD-ROM  as  part 
of  the  NAEP  Data  on  Disk  product  suite.  This  medium  can  be  ideal  for  researchers  and  policy  makers 
operating  in  a personal  computing  environment. 

The  NAEP  Data  on  Disk  product  suite  includes  two  other  components  which  facilitate  the 
analysis  of  NAEP  secondary-use  data.  The  PC-based  NAEP  data  extraction  software,  NAEPEX,  enables 
users  to  create  customized  extracts  of  NAEP  data  and  to  generate  SAS  or  SPSS  control  statements  for 
preparing  analyp  or  generating  customized  system  files.  The  NAEP  analysis  modules,  which  currently 
run  under  SPSS®  for  Windows™,  use  output  files  from  the  extraction  software  to  perform  analyses  that 
incorporate  statistical  procedures  appropriate  for  the  NAEP  design. 
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Chapter  9 


OVERVIEW  OF  PART  II: 

THE  ANALYSIS  OF  1996  NAEP  DATA^ 

Nancy  L.  Allen  and  James  E.  Carlson 
Educational  Testing  Service 


9.1  INTRODUCTION 

The  purpose  of  this  chapter  is  to  summarize  some  information  from  previous  chapters  that  is 
integral  to  the  analysis  of  NAEP  data,  to  summarize  the  analysis  steps  used  for  all  subjects,  and  to 
indicate  what  information  is  in  each  of  the  remaining  chapters.  The  overview  of  the  analyses  conducted 
on  the  1996  NAEP  data  focuses  on  the  common  elements  of  the  analyses  used  across  the  subject  areas  of 
the  assessment.  Some  of  this  information  is  available  only  within  this  chapter.  Details  by  subject  area  are 
provided  in  Chapters  12  through  17. 

The  organization  of  this  chapter  is  as  follows: 

• Section  9.2  provides  a short  overview  of  the  NAEP  design  for  1996.  To  provide 
additional  background  information,  the  section  also  provides  a short  description  of 
the  samples  selected  for  1996.  Chapters  1 through  8 provide  this  same  information  in 
much  more  detail. 

• Section  9.3  summarizes  the  steps  in  analysis  common  to  all  subject  areas.  Some  of 
this  information  is  described  in  more  detail  in  other  chapters.  The  rest  is  included 
only  within  this  chapter.  The  topics  covered  are  as  follows: 

• Section  9.3.1  briefly  describes  the  preparation  of  the  final  sampling  weights. 

Detailed  information  about  the  weighting  procedures  is  given  in  Chapter  10.  Detailed 
information  about  the  sampling  design  is  in  Chapter  3. 

• Section  9.3.2  provides  information  about  the  scoring  reliability  of  constructed- 
response  items.  It  provides  information  about  the  reliability  measures  used  with  the 
NAEP  data  during  analysis.  Chapter  7 contains  information  about  the  reliability 
procedures  used  during  the  scoring  process. 

• Section  9.3.3  summarizes  the  information  provided  by  the  teacher  questionnaires, 
and  indicates  its  use  during  the  analysis  process. 

• Section  9.3.4  provides  a description  of  the  item  properties  examined  for  background 
questions  and  for  cognitive  items.  It  includes  a description  of  the  classical  item 
statistics  examined  for  both  dichotomously  and  polytomously  scored  items.  It  also 


' Nancy  L.  Allen  was  responsible  for  the  psychometric  and  statistical  analyses  of  national  and  state  NAEP  data.  James  E.  Carlson 
was  responsible  for  psychometric  and  statistical  analyses  relating  to  special  aspects  and  issues  of  NAEP.  Eugene  G.  Johnson, 
John  Mazzeo,  Spencer  S.  Swinton,  and  Rebecca  Zwick  also  contributed  to  this  chapter. 
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includes  a description  of  the  item-level  results  available  from  summary  data  tables. 
Chapter  18  contains  more  information  about  the  conventions  used  in  creating  these 
summary  tables.  Finally,  a thorough  description  of  differential  item  functioning 
analyses  is  provided. 

• Section  9.3.5  summarizes  the  steps  used  to  scale  NAEP  data.  The  steps  include  IRT 
scaling  of  the  items,  generation  of  plausible  values  (conditioning),  transforming  the 
results  to  the  final  reporting  scale,  creating  composite  scores  if  necessary,  and 
providing  tables  of  reported  statistics.  Details  of  the  theory  behind  these  steps  are 
available  in  Chapter  1 1 . 

• Section  9.3.6  provides  some  information  about  previous  results  of  dimensionality 
analyses. 

• Finally,  Section  9.3.7  gives  an  introduction  to  hypothesis  testing  and  drawing  correct 
conclusions  about  NAEP  data.  Specific  information  about  which  hypothesis  test 
procedures  were  used  for  different  purposes  is  provided  in  Chapter  18. 

• Section  9.4  contains  a description  of  the  information  provided  in  Chapters  10 
through  19  of  this  report. 


9.2  SUMMARY  OF  THE  NAEP  DESIGN 

As  described  in  Chapter  1,  the  1996  NAEP  comprised  three  major  components.  One  component 
encompassed  major  assessments  in  mathematics  and  science,  providing  detailed  information  about 
student  proficiency  at  the  fourth-,  eighth-,  and  twelfth-grade  levels  of  nonpublic  and  public  schools. 
Long-term  trend  assessments  of  science,  mathematics,  and  reading  at  ages  9,  13,  and  17,  and  the  long- 
term trend  writing  assessment  for  grades  4,  8,  and  1 1,  constituted  the  second  component.  The  third  major 
component  was  the  State  Assessment  at  the  fourth-  and  eighth-grade  levels  in  mathematics  and  at  the 
eighth-grade  level  in  science.  Technical  details  of  the  State  Assessments  are  not  included  in  this  report 
but  are  presented  in  the  Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Mathematics 
(Allen,  Jenkins,  Kulick,  & Zelenak,  1997)  and  the  Technical  Report  of  the  NAEP  1996  State  Assessment 
Program  in  Science  (Allen,  Swinton,  Isham,  & Zelenak,  1997). 


In  addition  to  the  three  major  components,  special  studies  of  advanced  mathematics  students  at 
the  eighth-  and  twelfth-grade  levels  and  advanced  science  students  at  the  twelfth-grade  level  were 
conducted.  The  results  from  and  procedures  used  in  these  special  studies  are  reported  in  separate 
documents.  Likewise,  results  from  a study  of  holistic  scores  for  the  long-term  trend  writing  task 
responses  are  reported  in  a separate  document.  Results  based  on  primary  trait  scores  for  the  same  writing 
tasks  are  reported  in  the  NAEP  1996  Trends  in  Academic  Progress  (Campbell,  Voelkl,  & Donahue, 
1997),  and  the  analyses  contributing  to  those  results  are  described  in  Chapters  14-17  of  this  document. 
Finally,  results  for  the  items  associated  with  specific  mathematics  themes  are  reported  elsewhere. 

Results  from  the  analyses  described  in  the  following  chapters  were  reported  in  the  following 

reports: 


• The  NAEP  1996  Mathematics  Report  Card  for  the  Nation  and  the  States,  which 
provides  both  public-  and  nonpublic-school  data  for  major  NAEP  reporting 
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subgroups  for  all  of  the  jurisdictions  that  participated  in  the  State  Assessment 
program,  as  well  as  selected  results  from  the  1996  national  mathematics  assessment. 

• The  NAEP  1996  Science  Report  Card  for  the  Nation  and  the  States,  which  provides 
both  public-  and  nonpublic-school  data  for  major  NAEP  reporting  subgroups  for  all 
of  the  jurisdictions  that  participated  in  the  State  Assessment  program,  as  well  as 
selected  results  from  the  1996  national  science  assessment. 

• The  Cross-State  Data  Compendium  for  the  NAEP  1996  Mathematics  Assessment, 
which  includes  jurisdiction-level  results  for  all  the  demographic,  instructional,  and 
experiential  background  variables  included  in  the  Mathematics  Report  Card  and 
State  Reports. 

• The  Cross-State  Data  Compendium  for  the  NAEP  1996  Science  Assessment,  which 
includes  jurisdiction-level  results  for  all  the  demographic,  instructional,  and 
experiential  background  variables  included  in  the  Science  Report  Card  and  State 
Reports. 

• The  NAEP  1996  Trends  in  Academic  Progress,  which  looks  at  trends  in  average 
performance  over  time  in  the  areas  of  mathematics,  science,  reading,  and  writing. 


Because  the  samples  of  students  included  in  the  1996  NAEP  assessment  are  listed  and  described 
in  detail  in  Chapter  1,  only  a brief  description  of  these  samples  is  given  here.  The  1996  national  samples 
were  of  three  general  types:  main  NAEP  samples,  which  were  based  on  a common  set  of  assessment 
procedures,  including  grade-level  samples;  long-term  trend  samples,  the  purpose  of  which  was  to  provide 
links  to  earlier  assessments;  and  special  study  samples,  used  to  examine  results  for  advanced 
mathematics  and  science  students. 

To  shorten  the  timetable  for  reporting  results,  the  period  for  national  main  assessment  data 
collection  was  shortened  in  1992,  1994,  and  1996  from  the  five-month  period  (January  through  May) 
used  in  1990  and  earlier  assessments  to  a three-month  period  in  the  winter  (January  through  March, 
corresponding  to  the  period  used  for  the  winter  half-sample  of  the  1990  National  Assessment). 

The  1996  analyses  of  long-term  trend  data  extended  the  trend  lines  commencing  in  1971  in 
reading,  1973  in  mathematics,  1969  in  science,  and  1984  in  writing. 

As  described  in  Chapters  1 and  4,  for  each  subject  area  in  the  main  assessment,  blocks  of  items 
were  used  to  create  a large  number  of  different  assessment  booklets  according  to  a focused  design.  The 
1996  mathematics  assessment  used  a focused  balanced  incomplete  block  (focused-BIB)  design  while  the 
1996  science  assessment  used  a more  complex  design  due  to  the  inclusion  of  blocks  of  items  associated 
with  a specific  theme  or  hands-on  performance  task.  The  focused-BIB  design  provided  for  booklets  that 
typically  included  three  blocks  of  cognitive  items  in  a single  subject  area,  as  well  as  background  items. 
The  blocks  of  cognitive  items  for  mathematics  and  science  included  both  multiple-choice  and 
constructed-response  items.  In  a focused-BIB  design,  each  block  of  cognitive  items  appears  in  the  same 
number  of  booklets.  To  balance  possible  block  position  main  effects,  each  block  appears  an  equal 
number  of  times  in  each  position.  In  addition,  the  BIB  design  requires  that  each  block  of  items  be  paired 
in  a booklet  with  every  other  block  of  items  exactly  once. 
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9.3  ANALYSIS  STEPS 


Because  the  analysis  methods  are  not  identical  across  subject  areas  or  across  major  national  and 
long-term  trend  samples,  a separate  analysis  chapter  has  been  included  for  each  major  assessment  and  for 
each  long-term  trend  assessment.  The  procedures  used  depended  on  whether  assessment  items  were 
scored  dichotomously  (right  versus  wrong)  or  polytomously  (more  than  two  categories  of  response)  and 
whether  links  across  grade  levels  were  required.  Basic  procedures  common  to  most  or  all  of  the  subject 
area  analyses  are  summarized  here.  The  order  is  essentially  that  in  which  the  procedures  were  carried  out. 


9.3.1  Preparation  of  Final  Sampling  Weights 

Because  NAEP  uses  a complex  sampling  design  (Chapter  3)  in  which  students  in  certain 
subpopulations  have  different  probabilities  of  inclusion  in  the  sample,  the  data  collected  from  each 
student  must  be  assigned  a weight  to  be  used  in  analyses.  The  1996  NAEP  weights  were  provided  by 
Westat,  Inc.,  the  NAEP  contractor  in  charge  of  sampling.  Detailed  information  about  the  weighting 
procedures  is  available  in  Chapter  10  and  in  The  1996  NAEP  Sampling  and  Weighting  Report  (Wallace  & 


9.3.2  Reliability  of  Scoring  Constructed-Response  Items 

A minimum  of  25  percent  of  the  responses  for  science  items  involved  only  in  the  national 
assessment  and  six  percent  of  the  responses  for  mathematics  and  science  items  involved  in  both  the 
national  and  state  assessments  were  scored  by  a second  reader  to  obtain  statistics  on  interreader 
(interrater)  reliability.  Ranges  for  percentage  of  exact  agreement  for  state  and  national  assessments 
together,  of  mathematics  and  science  can  be  found  in  Table  9-1.  Average  percentage  of  exact  agreement 
for  each  booklet  type  (spiral  and  advanced)  can  be  found  in  Tables  7-5  and  7-6  in  Chapter  7.  This 
reliability  information  was  also  used  by  the  team  leaders  to  monitor  the  capabilities  of  all  readers  and 
maintain  uniformity  of  scoring  across  readers.  More  information  about  this  use  of  the  reliability 
information  is  in  Chapter  7. 


Table  9-1 

1996  Mathematics  and  Science  State  and  National  Assessments 
Ranges  of  Percentage  Exact  Agreement  Among  Readers 


Number  of 

Number  of  Items  in  Percentage 

Assessment 

Unique  Items 

Exact  Agreement  Range 

Total 

70-79% 

80-89% 

Above  90% 

Mathematics 

4'"  grade 

79 

0 

11 

68 

8'"  grade 

98 

1 

5 

92 

grade 

96 

1 

9 

86 

Science 

4"^  grade 

94 

0 

13 

81 

8"^  grade 

125 

0 

20 

105 

12"’  grade 

156 

0 

26 

130 

or- 
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In  addition  to  reliability  information  calculated  and  used  during  the  scoring  process,  several 
additional  reliability  measures  are  calculated  for  constructed-response  items  after  the  item  response  data 
has  been  placed  on  the  NAEP  database.  They  appear  in  Appendix  I.  These  include  a final  percentage 
exact  agreement,  the  intraclass  correlation,  Cohen’s  Kappa  (Cohen,  1968),  and  the  product-moment 
correlation  between  the  scores  for  the  first  and  second  readers.  These  measures  are  summarized  in  Zwick 
(1988),  Kaplan  and  Johnson  (1992),  and  Abedi  (1996).  Each  measure  has  advantages  and  disadvantages 
for  use  in  different  situations.  In  this  report,  the  percentage  exact  agreement  is  reported  for  all 
constructed-response  items,  Cohen’s  Kappa  is  reported  for  dichotomously  scored  constructed-response 
items,  and  the  intraclass  correlation  is  reported  for  polytomously  scored  constructed-response  items. 


9.3.3  Teacher  Questionnaires 

Teachers  of  students  who  were  in  the  fourth-  and  eighth-grade  mathematics  and  science  main 
assessment  samples  and  twelfth-grade  advanced  mathematics  samples  were  asked  to  complete  a two-part 
questionnaire.  The  first  part  of  the  questionnaire  pertained  to  the  teacher’s  background  and  training 
(Parts  I and  HA  in  Chapter  2).  The  second  part  pertained  to  the  procedures  used  by  the  teacher  for 
specific  classes  containing  assessed  students  (Parts  IIB  I Chapter  2).  See  Chapter  2 for  a description  of 
the  teacher  questionnaires. 

To  analyze  the  data  from  the  teacher  questionnaires  with  respect  to  the  students’  data,  each 
teacher’s  questionnaire  had  to  be  matched  to  all  of  the  sampled  students  who  were  taught  by  that  teacher. 
In  the  subsequent  chapters  two  separate  match  rates  for  each  grade  are  given.  The  first  is  the  percentage 
of  students  that  could  be  matched  to  both  the  first  and  second  parts  of  the  teacher  questionnaire.  For  these 
students,  information  is  available  not  only  about  the  background  and  training  of  their  teachers,  but  also 
about  the  methods  used  in  the  particular  class  they  attended.  The  second  match  rate  is  the  percentage  of 
students  that  could  be  matched  to  the  first  part  of  the  teacher  questionnaire.  This  match  rate  is  larger 
because  more  students  could  be  matched  with  information  about  a teacher  than  with  information  about 
the  particular  class  they  attended.  Note  that  these  match  rates  only  reflect  the  student-level  missing  data. 
They  do  not  reflect  the  additional  missing  data  due  to  item-level  nonresponse  on  the  part  of  teachers. 
Variables  derived  from  the  teacher  questionnaires  were  used  as  reporting  variables  at  the  student  level 
and  as  variables  that  contributed  to  conditioning  for  the  appropriate  samples. 

Teachers  of  students  who  were  in  the  grade  4 mathematics  main  assessment  sample  were  asked 
to  complete  a two-part  questionnaire.  As  with  the  grade  8 teacher  questionnaire,  the  first  part  pertained  to 
the  teacher’ s background  and  training.  Unlike  the  grade  8 teacher  questionnaire,  the  second  part 
pertained  to  only  a single  class  that  the  teacher  taught.  In  development  of  the  questionnaires,  it  was 
thought  that  fourth-grade  teachers  would  teach  one  class  in  each  subject.  In  practice,  that  was  found  to  be 
untrue  for  a number  of  teachers.  A single  student-teacher  match  rate  matching  students  to  the  first  part  of 
the  questionnaire  is  reported  for  grade  4 in  the  following  chapters. 


9.3.4  Analysis  of  Item  Properties:  Background  and  Cognitive  Items 

The  first  step  in  the  analysis  of  the  1996  data  was  item-level  analysis  of  all  instruments.  Item 
analyses  were  performed  separately  for  each  grade  or  age  level  on  each  item  in  each  subject  area.  Each 
block  of  items  was  analyzed  separately,  by  age  or  grade,  with  the  total  score  on  the  block  (including  the 
analyzed  item)  used  as  the  criterion  score  for  statistics  requiring  such  a score.  In  the  cases  where  final 
weights  were  not  available,  preliminary  weights  were  used  in  these  preliminary  analyses.  The  item 
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analysis  of  cognitive  items  was  repeated  after  scaling  of  the  items  was  completed.  The  results  for  only 
scaled  items  using  final  weights  are  reported  in  Chapters  12  through  17. 


O Background  Items 

For  each  background  item,  the  unweighted  and  weighted  percent  of  students  who  gave 

each  response  were  examined,  as  well  as  the  percent  of  students  who  omitted  the  item  and  the  percent 
who  did  not  reach  the  item.  The  number  of  respondents  was  also  tabulated.  These  preliminary  analyses 
were  conducted  within  age/grade  cohorts  and  within  major  reporting  categories.  If  unexpected  results 
were  found,  the  item  data  and  the  encoding  of  responses  were  rechecked. 


O Cognitive  Items 

All  NAEP  cognitive  items  were  subjected  to  analyses  of  item  properties.  These  analyses  included 
conventional  item  analyses  and  incorporated  examinee  sampling  weights.  Item  analysis  was  conducted  at 
the  block  level  so  that  the  “number-correct”  scores  for  students  responding  to  an  item,  selecting  each 
option  of  an  item,  omitting  an  item,  or  not  reaching  an  item,  is  the  average  number  of  correct  responses 
for  the  block  containing  that  item.  Because  of  the  inclusion  of  polytomously  scored  items  in  the  cognitive 
instruments,  it  was  necessary  to  use  special  procedures  for  these  items.  The  resulting  statistics  are 
analogous  to  those  for  the  dichotomously  scored  items,  as  listed  below. 


Dichotomously  Scored  Items.  These  items  were  analyzed  using  standard  procedures  that  result 
in  a report  for  each  item  that  includes: 

• for  each  option  of  the  item,  for  examinees  omitting  and  not  reaching  the  item,  and  for 
the  total  sample  of  examinees: 

=>  the  number  of  examinees, 

=>  the  percentage  of  examinees, 

^ the  mean  of  number-correct  scores  for  the  block  in  which  the  item  appears,  and 

^ the  standard  deviation  of  number-correct  scores  for  the  block  in  which  the  item 
appears; 

• the  percentage  of  examinees  providing  a response  that  was  “off-task;” 

• P+,  the  proportion  of  examinees  that  received  a correct  score  on  the  item  (ratio  of 
number  correct  to  number  correct  plus  wrong  plus  omitted); 

• A,  the  inverse-normally  transformed  p+  scaled  to  mean  13  and  standard  deviation  4; 

• the  biserial  correlation  coefficient  between  the  item  and  the  number-correct  scores 
for  the  block  in  which  the  item  appears;  and 

• the  point-biserial  correlation  coefficient  between  the  item  and  the  number-correct 
scores  for  the  block  in  which  the  item  appears. 
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Polytomously  Scored  Items.  Enhanced  procedures  were  employed  for  polytomously  scored 
items.  Methods  parallel  to  those  used  for  dichotomously  scored  items  resulted  in  values  reported  for  each 
distinct  response  category  for  the  item.  Response  categories  for  each  item  were  defined  in  two  ways,  one 
based  on  the  original  codes  for  responses  as  specified  in  the  scoring  rubrics  used  by  the  scorers  (and  the 
position  of  the  item  for  which  no  response  was  given)  and  one  based  on  a scoring  guide  developed  by 
subject  area  and  measurement  experts.  For  example,  a constructed-response  item  with  four  response 
categories  would  initially  have  seven  categories  (not-reached,  omitted,  “off -task,”  and  the  four  valid 
response  categories).  Another  set  of  statistics  resulted  from  mapping  the  response  categories  (excluding 
not-reached)  into  a new  set  of  categories  reflecting  the  scoring  guide  for  the  item.  A constructed-response 
item  with  ordered  categories,  for  example,  would  be  mapped  into  a set  of  integers  in  a corresponding 
order.  The  scoring  guide  could  result  in  the  collapsing  of  (combining  of)  some  response  categories.  The 
response  categories,  based  on  the  final  scoring  guide  developed  by  subject  area  and  measurement  experts, 
were  used  to  calculate  the  polytomously  scored  item  statistics. 

The  following  statistics,  analogous  to  those  for  dichotomously  scored  items,  were  computed: 

• the  percentage  of  examinees  providing  a response  that  was  “off-task;” 

• in  place  of  p-i-,  the  ratio  of  the  mean  item  score  to  the  maximum-possible  item  score 
was  used; 

• in  place  of  A,  the  ratio  of  the  mean  item  score  to  the  maximum-possible  item  score 
underwent  the  same  transformation  as  that  used  on  p-i-  to  get  A for  dichotomously 
scored  items; 

• the  polyserial  correlation  coefficient  was  used  in  place  of  the  biserial;  and 

• the  Pearson  correlation  coefficient  was  used  in  place  of  the  point-biserial. 


O Tables  of  Item-Level  Results 

Tables  were  created  of  the  percentages  of  students  choosing  each  of  the  possible  responses  to 
each  item  within  each  of  the  samples  administered  in  1996.  The  results  for  each  item  were  cross- 
tabulated  against  the  basic  reporting  variables  such  as  region,  gender,  race/ethnicity,  public/nonpublic 
school,  and  parental  education.  All  percentages  were  computed  using  the  sampling  weights.  These  tables 
are  referred  to  as  the  Test  Question  section  of  the  electronically  delivered  summary  data  tables  for  each 
sample  (see  Chapter  18  for  a brief  description  of  summary  data  tables).  In  the  summary  data  tables,  the 
sampling  variability  of  all  population  estimates  was  obtained  by  the  jackknife  procedure  used  by  ETS  in 
previous  assessments.  Details  of  these  procedures  are  presented  in  Chapter  10. 


O Differential  Item  Functioning  Analysis  of  Cognitive  Items 

Differential  item  functioning  (DIF)  analysis  refers  to  procedures  to  assess  whether  items  are 
differentially  difficult  for  different  groups  of  examinees.  DIF  procedures  typically  control  for  overall 
between-group  differences  on  a criterion,  usually  by  matching  examinees  from  the  two  groups  on  overall 
test  scores.  Between-group  performance  on  each  item  is  then  compared  within  sets  of  examinees  having 
the  same  total  test  scores. 
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DIF  analyses  were  conducted  for  items  in  the  main  assessments  in  mathematics  and  science  that 
had  not  previously  been  studied  for  differential  item  functioning.  Each  set  of  analyses  involved  three 
reference  group/focal  group  comparisons:  male/female,  White/Black,  and  White/Hispanic. 

The  DIF  analyses  of  the  dichotomous  items  were  based  on  the  Mantel-Haenszel  chi-square 
procedure,  as  adapted  by  Holland  and  Thayer  (1988).  The  procedure  tests  the  statistical  hypothesis  that 
the  odds  of  correctly  answering  an  item  are  the  same  for  two  groups  of  examinees  that  have  been 
matched  on  some  measure  of  proficiency  (usually  referred  to  as  the  matching  criterion).  The  DIF 
analyses  of  the  polytomous  items  were  based  on  the  Mantel  procedure  (Mantel,  1963).  These  procedures 
compare  proportions  of  matched  examinees  from  each  group  in  each  polytomous  item  response  category. 
The  groups  being  compared  are  often  referred  to  as  the  focal  group  (usually  a minority  or  other  group  of 
interest,  such  as  Black  examinees  or  female  examinees)  and  the  reference  group  (usually  White 
examinees  or  male  examinees). 

For  both  types  of  analyses,  the  measure  of  proficiency  used  is  typically  the  total  item  score  on 
some  collection  of  items.  Since,  by  the  nature  of  the  BIB  design,  booklets  comprise  different 
combinations  of  blocks,  there  is  no  single  set  of  items  common  to  all  examinees.  Therefore,  for  each 
student,  the  measure  of  proficiency  used  was  the  total  item  score  on  the  entire  booklet.  These  scores  were 
then  pooled  across  booklets  for  each  analysis.  This  procedure  is  described  by  Allen  and  Donoghue  (1994, 
1996).  In  addition,  because  research  results  (Zwick  & Grima,  1991)  strongly  suggest  that  sampling 
weights  should  be  used  in  conducting  DIF  analyses,  the  weights  were  used. 

For  each  dichotomous  item  in  the  assessment,  an  estimate  of  the  Mantel-Haenszel  common  odds- 
ratio,  expressed  on  the  ETS  delta  scale  for  item  difficulty,  was  produced.  The  estimates  indicate  the 
difference  between  reference  group  and  focal  group  item  difficulties  (measured  in  ETS  delta  scale  units), 
and  typically  run  between  about  +3  and  -3.  Positive  values  indicate  items  that  are  differentially  easier  for 
the  focal  group  than  the  reference  group  after  making  an  adjustment  for  the  overall  level  of  proficiency  in 
the  two  groups.  Similarly,  negative  values  indicate  items  that  are  differentially  harder  for  the  focal  group 
than  the  reference  group.  It  is  common  practice  at  ETS  to  categorize  each  item  into  one  of  three 
categories  (Petersen,  1988):  “A”  (items  exhibiting  no  DIF),  “B”  (items  exhibiting  a weak  indication  of 
DIF),  or  “C”  (items  exhibiting  a strong  indication  of  DIF).  Items  in  category  “A”  have  Mantel-Haenszel 
common  odds  ratios  on  the  delta  scale  that  do  not  differ  significantly  from  0 at  the  alpha  = .05  level  or 
are  less  than  1.0  in  absolute  value.  Category  “C”  items  are  those  with  Mantel-Haenszel  values  that  are 
significantly  greater  than  1 and  larger  than  1.5  in  absolute  magnitude.  Other  items  are  categorized  as  “B” 
items.  A plus  sign  (+)  indicates  that  items  are  differentially  easier  for  the  focal  group;  a minus  sign  (-) 
indicates  that  items  are  differentially  more  difficult  for  the  focal  group. 

The  ETS/NAEP  DIF  procedure  for  polytomous  items  uses  the  Mantel-Haenszel  ordinal 
procedure.  The  summary  tables  of  identified  polytomous  items  contain  generalizations  of  the 
dichotomous  “A,”  “B,”  and  “C”  categories:  “AA,”  “BB,”  or  “CC.” 

All  analyses  used  rescaled  sampling  weights.  A separate  rescaled  weight  was  defined  for  each 
comparison  as 


Rescaled  Weight  = Original  Weight  x 

Sum  of  the  Weights 

where  the  total  sample  size  is  the  total  number  of  students  for  the  two  groups  being  analyzed  (e.g.,  for  the 
White/Hispanic  comparison,  the  total  number  of  White  and  Hispanic  examinees  in  the  sample  at  that 
grade),  and  the  sum  of  the  weights  is  the  sum  of  the  sampling  weights  of  all  the  students  in  the  sample  for 
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the  two  groups  being  analyzed.  Three  rescaled  weights  were  computed  for  White  examinees  one  for  the 
gender  comparison  and  two  for  the  race/ethnicity  comparisons.  Two  rescaled  overall  weights  were 
computed  for  the  Black  and  Hispanic  examinees — one  for  the  gender  comparison  and  another  for  the 
appropriate  race/ethnicity  comparison.  The  rescaled  weights  were  used  to  ensure  that  the  sum  of  the 
weights  for  each  analysis  equaled  the  number  of  students  in  that  comparison,  thus  providing  an  accurate 
basis  for  significance  testing. 

In  the  calculation  of  total  item  scores  for  the  matching  criterion,  both  not-reached  and  omitted 
items  were  considered  to  be  wrong  responses.  Polytomous  items  were  weighted  more  heavily  in  the 
formation  of  the  matching  criterion,  proportional  to  the  number  of  score  categories.  For  each  item, 
calculation  of  the  Mantel-Haenszel  statistic  did  not  include  data  from  examinees  who  did  not  reach  the 
item  in  question. 

Each  DIF  analysis  was  a two-step  process.  In  the  initial  phase,  total  item  scores  were  formed,  and 
the  calculation  of  DIF  indices  was  completed.  Before  the  second  phase,  the  matching  criterion  was 
refined  by  removing  all  “C”  or  “CC”  items,  if  any,  from  the  total  item  score.  The  revised  score  was  used 
in  the  final  calculation  of  all  DIF  indices.  Note  that  when  analyzing  an  item  classified  as  “C”  or  “CC”  in 
the  initial  phase,  that  item  score  is  added  back  into  the  total  score  for  the  analysis  of  that  item  only. 

Following  standard  practice  at  ETS  for  DIF  analyses  conducted  on  final  forms,  all  “C”  or  “CC” 
items  were  reviewed  by  a committee  of  trained  test  developers  and  subject-matter  specialists.  Such 
committees  are  charged  with  making  judgments  about  whether  or  not  the  differential  difficulty  of  an  item 
is  unfairly  related  to  group  membership.  The  committee  assembled  to  review  NAEP  items  included  both 
ETS  staff  and  outside  members  with  expertise  in  the  field.  The  committee  carefully  examined  each 
identified  item  to  determine  if  either  the  language  or  contents  would  tend  to  make  the  item  more  difficult 
for  an  identified  group  of  examinees.  It  was  the  committee’s  judgment  that  none  of  the  “C”  or  “CC” 
items  in  the  national  assessment  were  functioning  differentially  due  to  factors  irrelevant  to  test 
objectives.  Hence,  none  of  the  items  were  removed  from  scaling  due  to  differential  item  functioning.  As 
pointed  out  by  Zieky  (1993): 

It  is  important  to  realize  that  DIF  is  not  a synonym  for  bias.  The  item  response  theory  based 
methods,  as  well  as  the  Mantel-Haenszel  and  standardization  methods  of  DIF  detection,  will  identify 
questions  that  are  not  measuring  the  same  dimension(s)  as  the  bulk  of  the  items  in  the  matching 
criterion.... Therefore,  judgment  is  required  to  determine  whether  or  not  the  difference  in  difficulty 
shown  by  a DIF  index  is  unfairly  related  to  group  membership.  The  judgment  of  fairness  is  based  on 
whether  or  not  the  difference  in  difficulty  is  believed  to  be  related  to  the  constmct  being 
measured....The  fairness  of  an  item  depends  directly  on  the  purpose  for  which  a test  is  being  used. 

For  example,  a science  item  that  is  differentially  difficult  for  women  may  be  judged  to  be  fair  in  a 
test  designed  for  certification  of  science  teachers  because  the  item  measures  a topic  that  every  entry- 
level  science  teacher  should  know.  However,  that  same  item,  with  the  same  DIF  value,  may  be 
judged  to  be  unfair  in  a test  of  general  knowledge  designed  for  all  entry-level  teachers,  (p.  340) 


9.3.5  Scaling 

Scales  based  on  item  response  theory  (IRT)  were  derived  for  each  subject  area.  A single  scale 
was  used  for  summarizing  long-term  trends  at  each  age  or  grade  level  in  each  of  the  subject  areas.  Five 
scales  were  created  for  mathematics  main  assessment  data,  one  for  each  mathematics  content  strand,  and 
three  scales  were  created  for  science  data,  one  for  each  field  of  science.  NAEP  uses  the  methodology  of 
multiple  imputations  (plausible  values)  to  estimate  characteristics  of  the  proficiency  distributions. 
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Chapter  1 1 describes  in  detail  the  theoretical  underpinnings  of  NAEP’s  scaling  methods  and  the  required 
estimation  procedures.  The  basic  analysis  steps  are  outlined  here. 

1.  Use  the  NAEP-BILOG/PARSCALE  computer  program  (described  in  Chapter  11)  to 
estimate  the  parameters  of  the  item  response  functions  on  an  arbitrary  provisional 
scale.  This  program  uses  an  IRT  model  incorporating  the  two-  and  three-parameter 
logistic  forms  used  in  previous  assessments  for  dichotomously  scored  items  and  the 
generalized  partial  credit  form  for  polytomously  scored  items.  In  order  to  select 
starting  values  for  the  iterative  parameter-estimation  procedure  for  each  dataset,  the 
program  is  first  run  to  convergence,  imposing  the  condition  of  a fixed  normal  prior 
distribution  of  the  proficiency  variable.  Once  these  starting  values  are  computed,  the 
main  estimation  runs  model  ability  as  a multinomial  distribution.  That  is,  no  prior 
assumption  about  the  shape  of  the  proficiency  distribution  is  made.  In  analyses 
involving  more  than  one  population,  estimates  of  parameters  are  made  with  the 
overall  mean  and  standard  deviation  of  all  subjects’  proficiencies  specified  to  be  0 
and  1,  respectively. 

2.  Use  a version  of  the  MGROUP  program  (described  in  Chapter  11),  which 
implements  the  method  of  Mislevy  (see  Chapter  11  or  Mislevy,  1991)  to  estimate 
predictive  proficiency  distributions  for  each  student  on  an  arbitrary  scale,  based  on 
the  item  parameter  estimates  and  the  student’s  responses  to  cognitive  items  and 
background  questions. 

3.  Use  random  draws  from  these  predictive  proficiency  distributions  (plausible  values, 
in  NAEP  terminology)  for  computing  the  statistics  of  interest,  such  as  mean 
proficiencies  for  demographic  groups. 

4.  Determine  the  appropriate  metric  for  reporting  the  results  and  transform  the  results 
as  needed.  This  includes  the  linking  of  current  scales  to  scales  from  the  past  or  the 
selection  of  the  mean  and  variance  of  new  scales.  After  proficiency  distributions  for 
the  scaling  are  transformed,  composite  proficiency  distributions  are  created  for  the 
mathematics  and  science  assessments. 

5.  Use  the  jackknife  procedure  to  estimate  the  standard  errors  of  the  mean  proficiencies 
for  the  various  demographic  groups. 

As  explained  in  Chapter  1 1,  the  plausible  values  obtained  through  the  IRT  approach  are  not 
optimal  estimates  of  individual  proficiency;  instead,  they  serve  as  intermediate  values  to  be  used  in 
estimating  subpopulation  characteristics.  Under  the  assumptions  of  the  scaling  models,  these 
subpopulation  estimates  are  statistically  consistent,  which  would  not  be  true  of  subpopulation  estimates 
obtained  by  aggregating  optimal  estimates  of  individual  proficiency. 


O Scaling  the  Cognitive  Items 

The  data  from  both  the  trend  and  the  main  assessment  samples  were  scaled  using  IRT  models. 
For  dichotomously  scored  items  two-  and  three-parameter  logistic  forms  of  the  model  were  used,  while 
for  polytomously  scored  items  the  generalized  partial  credit  model  form  was  used.  These  two  types  of 
items  and  models  were  combined  in  the  NAEP  scales.  Item  parameter  estimates  on  a provisional  scale 
were  obtained  using  the  NAEP  BILOG/PARSCALE  program.  The  fit  of  the  IRT  model  to  the  observed 
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data  was  examined  within  each  scale  by  comparing  the  empirical  item  response  functions  with  the 
theoretical  curves,  as  described  in  Chapter  11.  Plots  of  the  empirical  item  response  functions  and 
theoretical  curves  were  compared  across  assessments  for  long-term  trend  assessments.  The  DIF  analyses 
previously  described  provide  information  related  to  the  model  fit  across  subpopulations. 

Long-Term  Trend  Scaling.  Item  parameters  for  science,  mathematics,  reading,  and  writing 
trends  were  reestimated,  separately  for  each  age  or  age/grade  group  using  the  data  from  the  1994 
assessment  as  well  as  data  from  the  1996  assessment.  The  resulting  scales,  based  on  these  reestimated 
item  parameters,  were  then  linked  to  the  existing  long-term  trend  scales. 

Main  Assessment  Scaling.  The  main  assessments  of  mathematics  and  science  both  have  special 
characteristics  that  determine  the  procedures  that  were  followed  for  the  scaling  and  of  each  subject.  For 
mathematics,  a key  consideration  was  the  degree  of  similarity  between  the  1996  assessment  and  earlier 
assessments  in  terms  of  the  populations  assessed  and  the  characteristics  of  the  assessment  instrument 
used.  This  was  due  to  the  fact  that  the  mathematics  scales  were  linked  to  existing  mathematics  scales.  For 
science,  characteristics  of  items  associated  with  particular  themes  and  hands-on  performance  tasks  were 
of  concern.  The  science  scales  were  not  linked  to  any  previously  defined  scales. 

The  frameworks  for  the  different  subject  areas  dictate  differences  in  the  numbers  of  scales.  For 
mathematics  and  science,  item  parameter  estimation  was  performed  separately  for  each  of  several  scales 
defined  in  their  frameworks,  using  data  from  each  age/grade  sample  separately. 


«4>  Generation  of  Plausible  Values  for  Each  Scale 

After  the  scales  were  developed,  plausible  values  were  drawn  from  the  predictive  distribution  of 
proficiency  values  for  each  student  (this  process  is  called  conditioning).  For  the  long-term  trend  scales, 
the  plausible  values  were  computed  separately  for  each  age  or  age/grade  group  and  year,  and  were  based 
on  the  student’s  responses  to  the  items  going  into  the  scale  as  well  as  on  the  values  of  a set  of  background 
variables  that  were  important  for  the  reporting  of  proficiency  scores.  For  the  mathematics  and  science 
main  scales,  vectors  of  multivariate  plausible  values  were  drawn  from  the  joint  distribution  of 
proficiency  values  for  the  assessed  student.  These  multivariate  plausible  values  were  computed 
separately  for  each  grade  and  reflected  the  dependency  between  scale  proficiencies  by  utilizing  shared 
variation  among  the  scales.  All  plausible  values  were  later  rescaled  to  the  final  scale  metric  using 
appropriate  linear  transformations. 

The  variables  used  to  calculate  plausible  values  for  a given  main  assessment  scale  or  group  of 
scales  included  a broad  spectrum  of  background,  attitude,  and  experiential  variables  and  composites  of 
such  variables.  All  standard  reporting  variables  were  included.  To  enhance  numerical  stability  for  the 
main  assessment  scales,  the  original  background  variables  were  standardized  and  transformed  into  a set 
of  linearly  independent  variables  by  extracting  principal  components  from  the  correlation  matrix  of  the 
original  contrast  variables.  The  principal  components,  rather  than  the  original  variables,  were  used  as 
independent  variables  to  calculate  plausible  values  for  those  scales.  Trend  scales  used  the  same  or  similar 
sets  of  conditioning  variables  that  were  used  when  the  scales  were  originally  constructed.  Details  of  the 
conditioning  process  and  of  the  NAEP-BGROUP  and  NAEP-CGROUP  computer  programs  that 
implement  the  process  are  presented  in  Chapter  11. 
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O Transformation  to  the  Reporting  Metric 

Mathematics  short-term  trend  and  science,  mathematics,  reading,  and  writing  long-term  trend 
scales  were  linked  to  previous  assessment  scales  via  common  population  linking  procedures  described  in 
the  subject-specific  data  analysis  chapters.  Essentially,  the  1994  and  1996  data  were  calibrated  together. 
Data  from  the  two  assessments  were  scaled  together  in  the  same  BILOG/PARSCALE  run,  specifying  the 
samples  for  each  assessment  as  coming  from  different  populations.  For  each  scale,  the  mean  and  standard 
deviation  of  the  1994  data  from  this  joint  calibration  were  matched  to  the  mean  and  standard  deviation  of 
the  1994  data  as  previously  reported.  This  then  linked  the  1996  data  to  the  previously  established  scale. 
New  scales  were  established  for  the  science  main  assessment.  Then  the  metrics  for  the  newly  established 
science  scales  were  set  to  have  a mean  of  150  and  a standard  deviation  of  35. 

The  transformations  were  of  the  form 

^target  — A • ^calibrated  ^ 

where 


Qiargei  = scale  level  in  terms  of  the  system  of  units  of  the  final  scale  used  for 
reporting; 

Qcaiihraied  = scale  level  in  terms  of  the  system  of  units  of  the  provisional 
NAEP-BILOG/PARSCALE  scale; 


A 

B 


SD/a;.gg/  / SDcalihrated  » 
^^target  " ^ *^^calibrated  » 


^^largei  = the  estimated  or  selected  standard  deviation  of  the  proficiency 
distribution  to  be  matched; 


^^catihraied  = the  estimated  standard  deviation  of  the  sample  proficiency  distribution 
on  the  provisional  NAEP-BILOG/PARSCALE  scale; 

^target  = the  estimated  or  selected  mean  of  the  proficiency  distribution  to  be 
matched;  and 


^calibrated  = the  estimated  mean  of  the  sample  proficiency  distribution  on  the 
provisional  NAEP-BILOG/PARSCALE  scale. 

After  the  plausible  values  were  linearly  transformed  to  the  new  scale,  any  plausible  value  less  than  0 was 
censored  to  0.  For  all  1996  assessments  other  than  the  science  main  assessment,  any  value  greater  than 
500  was  censored  to  500;  for  the  science  main  assessment,  any  value  greater  than  300  was  censored  to 
300.  Fewer  than  1 percent  of  the  students  in  any  sample  were  censored  in  this  way.  The  final 
transformation  coefficients  for  transforming  each  provisional  scale  to  the  final  reporting  scale  are  given 
in  subsequent  chapters. 
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O Deflnition  of  Composites  for  the  Multivariate  Scales 

In  addition  to  the  plausible  values  for  each  scale,  composites  of  the  individual  mathematics  and 
science  main  assessment  scales  were  created  as  measures  of  overall  proficiency  within  these  subject 
areas.  These  composites  were  weighted  averages  of  the  plausible  values  of  the  individual  scales.  The 
weights  reflected  the  relative  importance  of  the  scales  and  were  provided  in  the  frameworks  developed 
by  the  subject  area  committees.  The  weights  are  approximately  proportional  to  the  number  of  items  in 
each  scale  at  a given  grade  level. 


O Tables  of  Proficiency  Means  and  Other  Reported  Statistics 

Proficiencies  and  trends  in  proficiencies  were  reported  by  age  or  grade  for  a variety  of  reporting 
categories.  Additionally,  for  the  main  assessments,  the  percentages  of  the  students  within  each  of  the 
reporting  groups  who  were  at  or  above  achievement  levels  were  reported  to  provide  information  about 
the  distribution  of  achievement  within  each  subject  area.  For  the  long-term  trend  assessments,  the 
percentages  of  the  students  within  each  of  the  reporting  groups  who  were  at  or  above  anchor  points  were 
reported  for  the  same  reason.  All  estimates  based  on  proficiency  values  have  reported  variances  or 
standard  errors  based  on  proficiency  values  including  the  error  component  due  to  the  latency  of 
proficiency  values  of  individual  students  as  well  as  the  error  component  due  to  sampling  variability. 
These  tables  are  part  of  the  electronically  delivered  summary  data  tables. 


9.3.6  Dimensionality  Analysis 

Over  the  years  a number  of  studies  have  been  conducted  in  order  to  seek  answers  to  the  question 
of  how  many  dimensions  underlie  the  various  NAEP  assessment  instruments,  and  whether  there  is  a 
sufficiently  strong  first  dimension  to  support  inferences  about  a composite  scale  in  subjects  such  as 
mathematics,  science,  and  reading.  In  addition,  for  the  1992  mathematics  and  reading  assessments,  a 
study  was  conducted  (Carlson,  1993)  to  determine  whether  the  increasing  emphasis  on  extended 
constructed-response  items  that  are  scored  polytomously  has  any  effect  on  the  dimensionality.  It  was 
determined  that  for  the  1992  NAEP  data,  item  type  was  not  related  to  any  of  the  dimensions  identified. 


O Previous  Dimensionality  Analyses  of  NAEP  Data 

In  an  early  study,  NAEP  reading  assessment  data  collected  during  the  1983-84  academic  year 
was  examined  for  dimensionality  by  Zwick  (1986,  1987).  Zwick  also  studied  simulated  data  designed  to 
mirror  the  NAEP  reading  item-response  data  but  having  known  dimensionality.  Analysis  of  the  simulated 
datasets  allowed  her  to  determine  whether  the  BIB  spiraling  design  artificially  increases  dimensionality. 
Zwick  found  substantial  agreement  among  various  statistical  procedures,  and  that  the  results  using  BIB 
spiraling  were  similar  to  results  for  complete  datasets.  Overall  she  concluded  that  “it  is  not  unreasonable 
to  treat  the  data  as  unidimensional  (1987,  p.  306).” 

Rock  (1991)  studied  the  dimensionality  of  the  NAEP  mathematics  and  science  tests  from  the 
1990  assessment  using  confirmatory  factor  analysis.  His  conclusion  was  that  there  was  little  evidence  for 
discriminant  validity  except  for  the  geometry  scale  at  the  eighth-grade  level,  and  that  “we  are  doing  little 
damage  in  using  a composite  score  in  mathematics  and  science  (p.  2).” 
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A second-order  factor  model  was  used  by  Muthen  (1991)  in  a further  analysis  of  Rock’s 
mathematics  data,  to  examine  subgroup  differences  in  dimensionality.  Evidence  of  content-specific 
variation  within  subgroups  was  found,  but  the  average  (across  seven  booklets)  percentages  of  such 
variation  was  very  small,  ranging  from  essentially  0 to  22,  and  two-thirds  of  these  percentages  were 
smaller  than  10. 

Carlson  and  Jirele  (1992)  examined  1990  NAEP  mathematics  data.  Analyses  of  simulated  one- 
dimensional data  were  also  conducted,  and  the  fit  to  these  data  was  slightly  better  than  that  to  the  real 
NAEP  data.  Although  there  was  some  evidence  suggesting  more  than  one  dimension  in  the  NAEP  data, 
the  strength  of  the  first  dimension  led  the  authors  to  conclude  that  the  data  “are  sufficiently  unidimen- 
sional to  support  the  use  of  a composite  scale  for  describing  the  NAEP  mathematics  data,  but  that  there  is 
evidence  that  two  dimensions  would  better  fit  the  data  than  one  (p.  3 1).” 

Carlson  (1993)  studied  the  dimensionality  of  the  1992  mathematics  and  reading  assessments.  The 
relative  sizes  of  fit  statistics  for  simulated  as  compared  to  actual  data  suggested  that  lack  of  fit  may  be 
more  due  to  the  BIB  spiraling  design  of  NAEP  than  the  number  of  dimensions  fitted.  Kaplan  (1995) 
similarly  found  that  the  chi-squared  goodness  of  fit  statistic  in  the  maximum  likelihood  factor  analysis 
model  was  inflated  when  data  were  generated  using  a BIB  design.  The  sizes  of  the  fit  statistics  for  incom- 
plete simulation  conditions  (a  BIB  design  as  in  the  actual  NAEP  assessment)  were  more  like  those  of  the 
real  data  than  were  those  of  the  case  of  simulation  of  a complete  data  matrix.  Consistent  with  findings  of 
Zwick  (1986,  1987),  however,  the  incomplete  design  for  data  collection  used  in  NAEP  does  not  appear  to 
be  artificially  inflating  the  number  of  dimensions  identified  using  these  procedures. 


9.3.7  Drawing  Inferences  from  the  Results 

Drawing  correct  inferences  from  the  results  of  the  assessments  depends  on  several  components. 
First,  the  hypothesis  of  no  difference  between  groups  must  be  tested  statistically.  For  the  1996 
assessment,  the  use  of  t-tests  was  introduced  for  most  comparisons.  These  tests  are  more  appropriate  than 
z-tests  based  on  normal  distribution  approximations  when  the  statistics  that  are  being  compared  are  from 
distributions  with  thicker  tails  than  those  from  the  normal  distribution.  The  statistical  significance  tests 
used  in  NAEP  are  described  in  detail  in  Chapter  18. 

A second  component  contributing  to  drawing  correct  inferences  is  the  way  in  which  error  rates 
are  controlled  when  multiple  comparisons  are  made.  If  we  wish  to  make  a number  of  comparisons  in  the 
same  analysis,  say  White  students  versus  Black,  Hispanic,  Asian/Pacific  Island,  and  American  Indian 
students,  the  probability  of  finding  “significance”  by  chance  for  at  least  one  comparison  increases  with 
the  family  size  or  number  of  comparisons.  By  the  Bonferroni  inequality,  for  a family  size  of  4,  for 
example,  the  probability  of  a false  positive  (Type  I error)  using  a = .05  is  less  than  or  equal  to  4 x .05 
=.20,  larger  than  most  decision-makers  would  accept. 

One  general  method  for  controlling  error  rates  in  multiple  comparisons  is  based  on  the 
Bonferroni  inequality.  In  this  method,  the  Bonferroni  inequality  is  applied  and  a is  divided  by  the  family 
size,  n.  Now  a = .05/4  = .0125,  and  using  a,  the  combined  probability  of  one  or  more  errors  in  the  four 
comparisons  remains  controlled  at  less  than  or  equal  to  .05.  Note  that  dividing  the  probability  by  n is  not 
the  same  as  multiplying  the  critical  value  or  the  confidence  band  by  n.  Indeed,  in  moving  from  a family 
size  of  1 to  4,  we  increase  the  critical  value  only  from  1.960  to  2.498,  a 27.4  percent  increase.  Doubling 
the  family  size  again,  to  8,  increases  the  critical  value  to  2.735,  an  additional  9.5  percent  increase.  To 
double  the  initial  critical  value  to  3.92,  the  family  size  would  have  to  be  increased  to  564. 
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The  power  of  the  tests  thus  depends  on  the  number  of  comparisons  planned.  There  may  be  cases 
for  which,  before  the  data  are  seen,  it  is  determined  that  only  certain  comparisons  will  be  conducted.  As 
an  example,  with  the  five  groups  above,  interest  might  lie  only  in  comparing  the  first  group  with  each  of 
the  others  (family  size  4),  rather  than  comparing  all  possible  pairs  of  groups  (family  size  10).  This  means 
that  some  possibly  significant  differences  will  not  be  found  or  discussed,  but  the  planned  comparisons 
will  have  greater  power  to  identify  real  differences  when  they  occur. 

In  1996,  several  other  methods  were  used  to  increase  the  power  of  statistical  tests.  For  cases 
where  comparisons  of  statistics  in  2 x 2 tables  were  made,  an  adapted  Bonferroni  procedure  varying  the 
family  size  for  each  consecutive  test  was  used  (Hochberg,  1988).  For  a very  large  number  of 
comparisons,  as  when  comparing  results  for  every  state  to  the  nation,  a different  criterion  was  used  to 
control  the  error  rates  due  to  the  large  number  of  comparisons.  This  criterion,  the  False  Discovery  Rate 
(FDR),  as  described  by  Benjamini  and  Hochberg  (1994),  contrasts  with  the  Familywise  Error  Rate 
(FWE)  criterion  used  in  the  Bonferroni  method.  For  trends  extending  over  several  administrations,  power 
is  gained  by  testing  least-squares  fitted  linear  and  quadratic  trends,  rather  than  individual  pairs  of  data 
points.  For  example,  if  the  linear  regression  coefficient  is  significantly  greater  than  0,  and  the  quadratic 
coefficient  is  not  different  from  0,  the  trend  over  time  is  positive,  even  though  the  Bonferroni  test  might 
declare  no  individual  pair  of  points  significantly  different.  These  ways  to  control  error  rates  in  multiple 
comparisons  are  described  in  Chapter  18. 

A third  component  contributing  to  drawing  correct  inferences  is  the  limitation  of  comparisons  to 
those  for  which  there  are  adequate  data.  In  NAEP  reports  and  data  summaries,  estimates  of  quantities 
such  as  composite  and  content  area  proficiency  means,  percentages  of  students  at  or  above  the 
achievement  levels,  and  percentages  of  students  indicating  particular  levels  of  background  variables  (as 
measured  in  the  student,  teacher,  and  school  questionnaires)  are  reported  for  the  total  population  as  well 
as  for  key  subgroups  determined  by  the  background  variables.  In  some  cases,  sample  sizes  were  not  large 
enough  to  permit  accurate  estimation  of  proficiency  and/or  background  variable  results  for  one  or  more 
of  the  categories  of  these  variables. 

For  results  to  be  reported  for  any  subgroup,  a minimum  sample  size  of  62  was  required.  This 
number  was  arrived  at  by  determining  the  sample  size  required  to  detect  an  effect  size  of  0.5  with  a 
probability  of  .8  or  greater.^  The  effect  size  of  0.5  pertains  to  the  “true”  difference  in  mean  proficiency 
between  the  subgroup  in  question  and  the  total  population,  divided  by  the  standard  deviation  of 
proficiency  in  the  total  population.  In  addition,  subgroup  members  must  represent  at  least  five  primary 
sampling  units  (PSUs). 

A fourth  component  contributing  to  drawing  correct  inferences  is  the  limitation  of  comparisons 
to  those  comparing  statistics  with  standard  errors  that  are  estimated  well.  Standard  errors  of  mean 
proficiencies,  proportions,  and  percentiles  play  an  important  role  in  interpreting  subgroup  results  and 
comparing  the  performances  of  two  or  more  subgroups.  The  jackknife  standard  errors  reported  by  NAEP 
are  statistics  whose  quality  depends  on  certain  features  of  the  sample  from  which  the  estimate  is 
obtained.  In  certain  cases,  typically  when  the  number  of  students  upon  which  the  standard  error  is  based 
is  small  or  when  this  group  of  students  all  come  from  a small  number  of  participating  schools,  the  mean 
squared  error  associated  with  the  estimated  standard  errors  may  be  quite  large.  In  the  summary  reports, 
estimated  standard  errors  subject  to  large  mean  squared  errors  are  followed  by  the  symbol  “!”. 


^ A design  effect  of  2 was  assumed  for  this  purpose,  implying  a sample  design-based  variance  twice  that  of  simple  random 
sampling.  This  is  consistent  with  previous  NAEP  experience  (Johnson  & Rust,  1992). 
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The  magnitude  of  the  mean  squared  error  associated  with  an  estimated  standard  error  for  the 
mean  or  proportion  of  a group  depends  on  the  coefficient  of  variation  (CV)  of  the  estimated  size  of  the 
population  group,  denoted  as  N.  This  coefficient  of  variation  is  estimated  by: 


CV(N)  = 


SE(N) 

N 


where  is  a point  estimate  of  N and  SE(^N^  is  the  jackknife  standard  error  of  N. 


Experience  with  previous  NAEP  assessments  suggests  that  when  this  coefficient  exceeds  0.2,  the 
mean  squared  error  of  the  estimated  standard  errors  of  means  and  proportions  based  on  samples  for  this 
group  may  be  quite  large.  Therefore,  the  standard  errors  of  means  and  proportions  for  all  subgroups  for 
which  the  coefficient  of  variation  of  the  population  size  exceeds  0.2  are  followed  by  “!”  in  the  tables  of 
all  summary  reports.  These  standard  errors,  and  any  confidence  intervals  or  significance  tests  involving 
these  standard  errors,  should  be  interpreted  with  caution.  (Further  discussion  of  this  issue  can  be  found  in 
Johnson  & Rust,  1992.) 

A final  component  contributing  to  drawing  correct  inferences  pertains  to  comparisons 
involving  extreme  proportions.  When  proportions  are  close  to  zero  or  one,  their  distributions  differ 
greatly  from  t-  or  z-distributions.  For  this  reason,  hypothesis  tests  of  the  sort  used  by  NAEP  are  not 
appropriate  in  these  cases.  Under  these  conditions,  no  test  is  made.  Chapter  18  includes  the  specific 
definition  of  extreme  proportion  used  in  the  analysis  of  1996  data. 


9.4  OVERVIEW  OF  CHAPTERS  10  THROUGH  19 

The  remaining  chapters  in  Part  II  of  this  report  are  as  follows: 

Chapter  10\  The  1996  National  Assessment  used  a stratified  multistage  probability  sampling 
design  that  provided  for  sampling  certain  subpopulations  at  higher  rates  (see  Chapter  3).  Because 
probabilities  of  selection  are  not  the  same  for  all  assessed  students,  sampling  weights  must  be  used  in  the 
analysis  of  NAEP  data.  Also,  in  NAEP’s  complex  sample,  observations  are  not  independent.  As  a result, 
conventional  formulas  for  estimating  the  sampling  variance  of  statistics  are  inappropriate.  Chapter  10 
describes  the  weighting  procedures  and  methods  for  estimating  sampling  variance  that  are  necessitated 
by  NAEP’s  sample  design.  Further  detail  on  sampling  and  weighting  procedures  is  provided  in  The  1996 
NAEP  Sampling  and  Weighting  Report,  published  in  1999  by  Westat,  Inc.,  the  NAEP  contractor  in 
charge  of  sampling. 

Chapter  11:  A major  NAEP  innovation  introduced  by  ETS  is  the  reporting  of  subject-area  results 
in  terms  of  IRT-based  scales.  Scaling  methods  can  be  used  to  summarize  results  even  when  students 
answer  different  subsets  of  items.  For  purposes  of  summarizing  item  responses,  NAEP  developed  a 
scaling  technique  that  has  its  roots  in  IRT  and  in  the  theories  of  imputation  of  missing  data.  Chapter  1 1 
describes  this  scaling  technique,  the  underlying  theory,  and  the  application  of  these  methods  to  1996 
NAEP  data.  The  final  section  of  Chapter  1 1 gives  an  overview  of  the  NAEP  scales  that  were  developed 
for  the  1996  assessment. 

Chapter  12:  The  main  short-term  trend  component  of  the  1996  mathematics  analysis  is  described 
in  this  chapter.  A detailed  analysis  of  the  main  assessment  of  mathematics  was  conducted  for  grades  4,  8, 
and  12,  including  a study  of  the  association  between  mathematics  proficiency  and  student  background 
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variables.  The  results  from  this  component  of  the  analysis  can  be  compared  with  results  frorri  the  1992 
mathematics  analysis  to  examine  short-term  trends.  At  grades  8 and  12,  background  information  and  data 
on  instructional  methods  were  collected  from  teachers  and  the  relation  of  these  variables  to  mathematics 
proficiency  was  examined.  The  main  assessment  analyses  are  reported  in  NAEP  1996  Mathematics 
Report  Card  for  the  Nation  and  the  States  (Reese,  Miller,  Mazzeo,  & Dossey,  1997).  A special  study  of 
advanced  eighth-  and  twelfth-grade  mathematics  students  was  conducted,  and  cognitive  item  responses 
associated  with  specific  mathematics  themes  for  the  main  mathematics  samples  was  studied. 

Chapter  13:  The  main  assessment  analysis  of  the  science  data  is  detailed  in  Chapter  13.  This 
analysis  included  a study  of  the  association  of  science  knowledge  with  instructional  techniques  and 
student  background  variables.  At  grade  8,  background  information  and  data  on  instructional  methods 
were  collected  from  teachers  and  the  relation  of  these  variables  to  science  proficiency  was  examined.  The 
science  results  appear  in  NAEP  1996  Science  Report  Card  for  the  Nation  and  the  States  (O’Sullivan, 
Reese,  & Mazzeo,  1997).  A special  study  of  advanced  twelfth-grade  science  students  was  also  conducted. 

Chapter  14:  The  reading  trend  results  for  the  years  1971  through  1994  were  extended  to  include 
1996  at  ages  9,  13,  and  17.  The  results  of  the  reading  trend  analysis,  which  include  the  percentages  of 
students  at  or  above  the  reading  scale  anchor  points  established  in  1984,  are  reported  in  NAEP  1996 
Trends  in  Academic  Progress:  Achievement  ofU.S.  Students  in  Science,  1969  to  1996;  Mathematics, 

1973  to  1996;  Reading,  1971  to  1996;  and  Writing,  1984  to  1996  (Campbell,  Voelkl,  & Donahue,  1997). 

Chapter  15:  The  long-term  trend  assessment  analysis  of  the  mathematics  data  is  detailed  in 
Chapter  15.  The  results  of  the  trend  analysis,  which  provided  links  from  1973  through  1996  for  ages  9, 

13,  and  l7,  are  reported  in  NAEP  1996  Trends  in  Academic  Progress:  Achievement  of  U.S.  Students  in 
Science,  1969  to  1996;  Mathematics,  1973  to  1996;  Reading,  1971  to  1996;  and  Writing,  1984  to  1996 
(Campbell,  Voelkl,  & Donahue,  1997). 

Chapter  16:  The  long-term  trend  assessment  analysis  of  the  science  data  is  described  in  Chapter 
16.  The  science  trend  results,  which  provide  a link  to  1970,  1973,  1977,  1982,  1986,  1990,  1992,  and 
1994  are  reported  in  NAEP  1996  Trends  in  Academic  Progress:  Achievement  of  U.S.  Students  in  Science, 
1969  to  1996;  Mathematics,  1973  to  1996;  Reading,  1971  to  1996;  and  Writing,  1984  to  1996 
(Campbell,  Voelkl,  & Donahue,  1997). 

Chapter  17:  Like  the  1996  reading  assessment,  the  writing  assessment  in  1996  consisted  of  only 
one  component,  a long-term  trend.  The  writing  trend  results,  which  provide  a link  to  1984,  1988,  1990, 
1992,  and  1994  for  grades  4,  8,  and  12,  are  reported  in  NAEP  1996  Trends  in  Academic  Progress: 
Achievement  of  U.S.  Students  in  Science,  1969  to  1996;  Mathematics,  1973  to  1996;  Reading,  1971  to 
1996;  and  Writing,  1984  to  1996  (Campbell,  Voelkl,  & Donahue,  1997). 

Chapter  18:  The  1996  assessment  analyses  included  changes  in  the  methods,  procedures,  and 
conventions  used  in  making  group  comparisons.  Chapter  18  highlights  these  changes  and  provides  details 
about  which  results  were  reported. 

Chapter  19:  This  chapter  presents  basic  data  from  the  1996  assessment,  including  the  properties 
of  the  measuring  instruments  and  characteristics  of  the  sample. 
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Chapter  10 


WEIGHTING  PROCEDURES  AND  ESTIMATION 
OF  SAMPLING  VARIANCE* 

Eugene  G.  Johnson  and  Jiahe  Qian 
Educational  Testing  Service 

Leslie  Wallace  and  Keith  F.  Rust 
Westat,  Inc. 


10.1  INTRODUCTION 

As  was  the  case  in  previous  assessments,  the  1996  national  assessment  used  a complex  sample 
design  with  the  goal  of  securing  a sample  from  which  estimates  of  population  and  subpopulation 
characteristics  could  be  obtained  with  reasonably  high  precision  (as  measured  by  low  sampling 
variability).  At  the  same  time,  it  was  necessary  that  the  sample  be  economically  and  operationally  feasible 
to  obtain.  The  resulting  sample  had  certain  properties  that  had  to  be  taken  into  account  to  ensure  valid 
analyses  of  the  data  from  the  assessment. 

The  1996  NAEP  sample  was  obtained  through  a stratified  multistage  probability  sampling  design 
that  included  provisions  for  sampling  certain  subpopulations  at  higher  rates  (see  Chapter  3).  To  account  for 
the  differential  probabilities  of  selection,  and  to  allow  for  adjustments  for  nonresponse,  each  student  was 
assigned  a sampling  weight.  Section  10.2  discusses  the  procedures  used  to  derive  these  sampling  weights. 

Another  consequence  of  the  NAEP  sample  design  is  its  effect  on  the  estimation  of  sampling 
variability.  Because  of  the  effects  of  cluster  selection  (students  within  schools,  schools  within  primary 
sampling  units)  and  because  of  the  effects  of  certain  adjustments  to  the  sampling  weights  (nonresponse 
adjustment  and  poststratification),  observations  made  on  different  students  cannot  be  assumed  to  be 
independent  of  one  another.  In  particular,  as  a result  of  clustering,  ordinary  formulas  for  the  estimation  of 
the  variance  of  sample  statistics,  based  on  assumptions  of  independence,  will  tend  to  underestimate  the 
true  sampling  variability.  Section  10.3  discusses  the  jackknife  technique  used  by  NAEP  to  estimate 
sampling  variability.  (The  estimation  of  variability  due  to  imperfect  measurement  of  individual  proficiency 
is  discussed  in  Chapter  11.) 

The  jackknife  technique  provides  good  quality  estimates  of  sampling  variability  but  requires 
considerable  computations.  Section  10.4  suggests  the  use  of  design  effects,  combined  with  conventional 
variance  estimation  formulas,  as  a simple  approximation  to  sampling  variability  estimation. 

Since  the  sample  design  determines  the  derivation  of  the  sampling  weights  and  the  estimation  of 
sampling  variability,  it  will  be  helpful  to  note  the  key  features  of  the  1996  NAEP  sample  design.  A 
description  of  the  design  appears  in  Chapter  3,  and  the  various  assessment  instruments  are  detailed  in 
Chapter  4. 


' Keith  F.  Rust,  and  Leslie  Wallace  were  responsible  for  the  design  and  implementation  of  the  weighting  process  for  the  1996 
NAEP  assessments.  Previous  versions  of  this  chapter  were  created  with  the  significant  contributions  of  Eugene  Johnson, 
Educational  Testing  Service.  Jiahe  Qian  of  Educational  Testing  Service  made  significant  contributions  to  the  sampling  variability 
sections  of  this  chapter.  The  statistical  programming  for  this  chapter  was  overseen  by  Bruce  Kaplan  and  provided  by  Phillip  Leung. 
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The  1996  sample  was  a multistage  probability  sample  consisting  of  four  stages  of  selection  for  the 
long-term  trend  samples  and  five  stages  of  selection  for  the  main  samples.  The  first  stage  of  selection,  the 
primary  sampling  units  (PSUs),  consisted  of  counties  or  groups  of  counties.  The  second  stage  of  selection 
consisted  of  elementary  and  secondary  schools.  For  the  long-term  trend  assessment,  the  assignment  of 
sessions  to  sampled  schools  comprised  the  third  stage  of  sampling,  and  the  fourth  stage  involved  the 
selection  of  students  within  schools  and  their  assignment  to  sessions.  For  the  main  assessment,  the 
assignment  of  schools  to  sample  type  (see  Chapter  3)  comprised  the  third  stage  of  sampling,  the 
assignment  of  sessions  to  sampled  schools  comprised  the  fourth  stage  of  sampling,  and  the  fifth  stage 
involved  the  selection  of  students  within  schools  and  their  assignment  to  sessions. 

The  probabilities  of  selection  of  the  first-stage  sampling  units  were  proportional  to  measures  of 
their  size,  while  the  probabilities  for  subsequent  stages  of  selection  were  such  that  the  overall  probabilities 
of  selection  of  students  were  approximately  uniform,  with  exceptions  for  certain  subpopulations  that  were 
oversampled  by  design.  For  the  main  assessment,  schools  with  relatively  high  concentrations  of  Black 
students  and/or  Hispanic  students  were  deliberately  sampled  at  twice  the  normal  rate  to  obtain  larger 
samples  of  respondents  from  those  subpopulations,  in  order  to  increase  the  precision  in  the  estimation  of 
the  ch^acteristics  of  these  subpopulations.  Also  for  the  main  assessment,  nonpublic  school  students  were 
sampled  at  three  times  the  normal  rate,  again  to  increase  the  precision  of  estimates  for  this  population 
subgroup.  For  all  assessment  components,  students  from  schools  with  smaller  numbers  of  eligibles 
received  lower  probabilities  of  selection,  as  a means  of  enhancing  the  cost  efficiency  of  the  sample. 

The  1996  main  assessment  includes  three  student  cohorts;  students  in  grades  4,  8,  and  12.  The 
main  assessment  of  all  grades  was  conducted  in  the  spring  of  . 1996  to  provide  a cross-sectional  view  of 
students’  abilities  in  mathematics  and  science. 

The  1996  assessment  also  included  a number  of  additional  samples  that  used  the  age  definitions, 
times  of  testing,  and  modes  of  administration  used  in  previous  assessments.  These  are  referred  to  as  long- 
term trend  samples.  The  purpose  of  these  samples  was  to  provide  the  statistical  linkage  between  the  1996 
data  and  data  from  previous  assessments.  The  long-term  trend  assessment  represents  two  overlapping 
samples  of  students,  the  first  of  specified  grades  (of  any  age)  and  the  second  of  specified  ages  (in  any 
grade).  Students  were  age-eligible  if  they  were  bom  in  the  appropriate  year  (1986,  1982,  or  October  1978 
to  September  1979).  The  corresponding  grades  for  the  long-term  trend  assessment  were  4,  8,  and  11.  Each 
student  cohort  is  called  an  “age  class.” 

The  full  1996  NAEP  assessment  thus  includes  a number  of  different  samples  from  several 
populations.  Each  of  these  samples  has  its  own  set  of  weights  that  are  to  be  used  to  produce  estimates  of 
the  characteristics  of  the  population  addressed  by  the  sample  (the  target  population).  Each  main  sample  has 
an  additional  set  of  weights  to  accommodate  the  reporting  requirements.  The  various  samples  and  their 
target  populations  are  as  follows; 

The  Main  Samples  of  Students.  The  target  population  for  each  of  these  samples  (one  for  each 
grade)  consisted  of  all  students  who  were  in  the  specified  grade  and  were  deemed  assessable  by  their 
school.  There  were  four  distinct  session  types  at  grade  4 (mathematics,  science,  mathematics  estimation, 
and  mathematics  theme),  five  at  grade  8 (mathematics,  science,  mathematics  estimation,  mathematics 
theme,  advanced  mathematics),  and  six  at  grade  12  (mathematics,  science,  mathematics  estimation, 
mathematics  theme,  advanced  mathematics,  and  advanced  science).  Each  session  type  was  conducted  as 
one  or  more  distinct  sessions  within  a school.  Administration  of  each  session  type  was  always  conducted 
separately  from  other  session  types. 

To  facilitate  analyses,  two  kinds  of  weights  were  produced.  “Reporting  weights”  were  produced 
separately  by  grade  and  assessment  type  for  analyses  of  the  reporting  samples  that  were  defined  for  each 
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assessment.  Several  of  the  reporting  samples  included  students  from  multiple  sample  types.  “Modular 
weights”  were  produced  separately  by  grade,  assessment  type,  and  sample  type,  for  analyses  involving  any 
one  sample  type,  or  for  comparing  one  sample  type  with  another.  Thus  in  total,  across  grades,  session 
types,  and  sample  types,  there  were  15  sets  of  reporting  weights  and  32  sets  of  modular  weights  for 
students  in  the  main  assessment. 

Long-Term  Trend  Samples  for  Reading  and  Writing.  These  consist  of  samples  comparable  to  the 
1984  main  assessment  and  address  the  subject  areas  of  reading  and  writing.  The  samples  were  collected  by 
grade  and  age  for  age  9/grade  4,  age  13/grade  8,  and  age  17/grade  11,  using  the  age  definitions  and  time  of 
testing  from  1984.  As  in  that  assessment,  print  administration  was  used.  Six  assessment  booklets  were 
administered  at  each  age  class.  The  respondents  to  the  combined  set  of  assigned  booklets  at  a given  age 
class  constitute  a representative  sample  of  the  population  of  students  who  were  in  the  specified  grade  or  of 
the  specified  age.  The  respondents  to  any  one  of  the  booklets  also  constitute  a representative  sample. 

Long-Term  Trend  Samples  for  Mathematics  and  Science.  These  consist  of  samples  comparable 
to  those  used  for  the  measurement  of  trends  in  1986.  The  samples  were  collected  by  age  only  and  using  the 
same  age  definitions  and  time  of  testing  as  in  the  long-term  trend  assessment  in  1986.  As  in  that 
assessment,  the  administration  of  mathematics  and  science  questions  was  paced  with  2m  audiotape.  For 
ages  9 and  13,  three  assessment  booklets  were  administered  to  each  age  group  while  two  booklets  were 
administered  at  age  17.  The  respondents  to  any  one  of  the  booklets  assigned  to  a given  age  constitute  a 
representative  sample  of  the  population  of  all  students  of  that  age.  Each  booklet  was  administered  in  a 
separate  assessment  session,  but  the  booklets  were  combined  for  weighting  and  reporting. 

For  purposes  of  sampling  and  weighting,  the  assessment  samples  are  categorized  as  “tape- 
administered”  or  “print-administered”  according  to  if  paced  audiotapes  were  used  in  the  administration: 

1.  Tape-administered  samples  are  samples  that  required  audiotape  pacing  in  the 
assessment  (the  long-term  trend  assessments  in  mathematics  and  science).  For  these 
samples,  all  students  within  a particular  assessment  session  received  the  same  booklet 
and  were  paced  through  at  least  part  of  the  booklet  with  an  audiotape. 

2.  Print-administered  samples  are  all  main  assessment  samples  and  the  long-term  trend 
assessments  of  reading  and  writing.  For  these  samples,  no  audiotape  pacing  was 
employed  and  the  assessment  booklets  were  spiraled  through  each  assessment  session 
(that  is,  the  different  booklets  that  were  part  of  a given  session  type  were 
systematically  interspersed  and  assigned  for  testing  in  that  order). 


10.2  DERIVATION  OF  THE  SAMPLE  WEIGHTS 

As  indicated  previously,  NAEP  uses  differential  sampling  rates,  deliberately  oversampling  certain 
subpopulations  to  obtain  larger  samples  of  respondents  from  those  subgroups,  thereby  enhancing  the 
precision  of  estimates  of  characteristics  of  these  oversampled  subgroups.  As  a result  of  the  oversampling, 
these  subpopulations,  corresponding  to  students  from  schools  with  high  concentrations  of  Black  and/or 
Hispanic  students,  and  from  nonpublic  schools,  are  overrepresented  in  the  sample.  Lower  sampling  rates 
were  introduced  also  for  very  small  schools  (those  schools  with  only  1 to  19  eligible  students).  This 
reduced  level  of  sampling  from  small  schools  was  undertaken  in  an  approximately  optimum  manner  as  a 
means  of  reducing  variances  per  unit  of  cost  (since  it  is  relatively  costly  to  administer  assessments  in  these 
small  schools).  Appropriate  estimation  of  population  characteristics  must  take  disproportionate 
representation  into  account.  This  is  accomplished  by  assigning  a weight  to  each  respondent,  where  the 
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weights  approximately  account  for  the  sample  design  and  reflect  the  appropriate  proportional 
representation  of  the  various  types  of  individuals  in  the  population. 

Two  sets  of  weights  were  computed  for  the  1996  main  samples  for  each  session  type.  “Modular 
weights”  were  computed  for  analyses  involving  students  in  one  sample  type,  or  for  comparing  results 
between  sample  types.  Each  assessment  type  by  grade  and  sample  type  weights  up  separately  to  the  target 
population.  “Reporting  weights”  were  computed  for  analyses  of  the  reporting  samples  defined  in  Table  10- 
1 . Many  of  the  reporting  samples  include  students  from  more  than  one  sample  type.  For  reporting  samples 
that  include  only  one  sample  type  (i.e.,  science),  the  reporting  weights  are  identical  to  the  modular  weights. 
The  steps  for  computing  these  two  sets  of  weights  are  identical,  up  to  and  including  the  step  of  “trimming” 
the  weights.  The  trimmed  weights  were  poststratified  separately  by  sample  type  to  create  the  modular 
weights.  In  a parallel  procedure,  the  trimmed  weights  were  scaled  back  using  a “reporting  factor”  so  that 
the  sample  types  included  in  each  reporting  sample,  when  combined,  would  weight  up  to  the  target 
population.  The  resulting  weights  were  poststratified  (but  not  separately  by  sample  type)  to  create  the 
reporting  weights. 


Table  10-1 

Reporting  Samples  for  the  1996  Main  NAEP  Samples 


Assessment  Type 

Grade 

Reporting  Sample' 

Mathematics 

All 

A1  + A2  + B1 

Science 

All 

A2  + B2 

Mathematics  Estimation 

4 or  8 

Al  + Bl^ 

12 

A1  + A2  + Bl^ 

Mathematics  Theme 

All 

A2  + B2^ 

Advanced  Mathematics 

8 

A2  + A3  + B2 

12 

A3  + B3 

Advanced  Science 

12 

A2  + B2 

* A indicates  assessed  non  SD/LEP  students,  B indicates  assessed  SD/LEP  students;  and  1,  2, 
or  3 indicates  the  sample  type  (see  Chapter  3). 

^ In  the  actual  data  analysis,  the  1996  Mathematics  Estimation  used  the  reporting  samples  of 
A1  + A3  + B1  for  4th  and  8th  grades,  and  the  reporting  samples  of  A1  + A2  + A3  + B 1 + B2 
for  12th  grade.  The  weights  used  in  the  data  analysis  were  adjusted  based  on  the  information 
of  the  target  population  of  the  non-SD/LEP  and  SD/LEP  students  separately. 

^ In  the  actual  data  analysis,  the  1996  Mathematics  Theme  used  reporting  samples  of  A2  + A3 
+ B2.  The  weights  used  in  the  data  analysis  were  adjusted  based  on  the  information  of  the 
target  population  of  the  non-SD/LEP  and  SD/LEP  students  separately. 


The  weighting  procedures  for  1996  included  computing  the  student’s  base  weight,  the  reciprocal 
of  the  probability  that  the  student  was  selected  for  a particular  session  type.  Such  weights  are  those 
appropriate  for  deriving  estimates  from  probability  samples  via  the  standard  Horvitz-Thompson  estimator 
(see  Cochran,  1977).  These  base  weights  were  adjusted  for  nonresponse  and  then  subjected  to  a trimming 
algorithm  to  reduce  a few  excessively  large  weights.  The  weights  were  further  adjusted  by  a student-level 
poststratification  procedure  to  reduce  the  sampling  error.  This  poststratification  was  performed  by 
adjusting  the  weights  of  the  sampled  students  so  that  the  resulting  estimates  of  the  total  number  of  students 
in  a set  of  specified  subgroups  of  the  population  corresponded  to  population  totals  based  on  information 
from  the  Current  Population  Survey  and  U.S.  Census  Bureau  estimates  of  the  population.  The 
subpopulations  were  defined  in  terms  of  race,  ethnicity,  geographic  region,  grade,  and  age  relative  to 
grade. 


200 


223 


In  addition,  the  weights  at  grade  8 were  poststratified  at  the  school  level  in  an  effort  to  correct  an 
imbalance  in  the  school  sample  that  was  observed  after  the  assessments  were  complete.  This  procedure  has 
not  been  used  in  prior  NAEP  assessments,  and  was  introduced  because  at  grade  8,  the  three  sample  types 
contained  relatively  different  proportions  of  schools  with  large  numbers  of  Black  and  Hispanic  students 
(termed  “high  minority”  schools).  The  different  school  distributions  became  apparent  by  reviewing  student 
distributions  by  sample  type  and  race/ethnicity  for  the  mathematics  assessment.  In  mathematics  at  grade  8, 
sample  type  2 contained  a larger  proportion  of  Black  students  than  sample  type  1.  Investigation  revealed 
that  the  differences  in  student  distributions  were  due  to  different  school  distributions  by  percent  minority 
enrollment.  Although  the  different  proportions  were  within  sampling  error,  a decision  was  made  to  adjust 
the  weights  at  grade  8 for  each  of  the  sample  types  separately  so  that  in  each  case  the  resulting  estimates  of 
number  of  schools  by  percent  minority  enrollment  corresponded  to  totals  based  on  information  from  the 
sampling  frame  of  schools. 

The  following  sections  provide  an  overview  of  the  procedures  used  to  derive  the  sampling  weights. 
Further  details  in  the  derivation  of  these  weights  can  be  found  in  The  1996  NAEP  Sampling  and  Weighting 
Report  (Wallace  & Rust,  1999). 


10.2.1  Derivation  of  Reporting  Weights  for  the  Main  Samples 

Separate  weights  were  computed  for  each  assessment  in  the  main  samples  (mathematics,  science, 
mathematics  theme,  mathematics  estimation,  advanced  mathematics,  and  advanced  science).  In  earlier 
NAEP  assessments,  excluded  students  from  all  subjects  were  weighted  together,  separately  from  assessed 
students.  In  1996,  excluded  students  were  weighted  with  assessed  students  for  each  assessment.  This 
change  in  procedures  was  adopted  because  evidence  indicated  that  exclusion  rates  may  not  be  independent 
of  session  type,  and  because  this  change  simplified  the  weighting  procedures,  when  accounting  for  sample 
type.  Reporting  weights  for  the  main  samples  were  derived  according  to  the  steps  outlined  below. 


10.2.1.1  Student  Base  Weight 

The  base  weight  assigned  to  a student  is  the  reciprocal  of  the  probability  that  the  student  was 
selected  for  a particular  assessment.  That  probability  is  the  product  of  five  factors: 

1 . the  probability  that  the  PSU  was  selected; 

2.  the  conditional  probability,  given  the  PSU,  that  the  school  was  selected; 

3.  the  conditional  probability,  given  the  sample  of  schools  in  a PSU,  that  the  school  was 
assigned  the  specified  sample  type; 

4.  the  conditional  probability,  given  the  sample  of  schools  in  a PSU,  that  the  school  was 
allocated  the  specified  session  type;  and 

5.  the  conditional  probability,  given  the  school,  that  the  student  was  selected  for  the 
specified  session  type. 
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Thus,  the  base  weight  for  a student  may  be  expressed  as  the  product 


Ws  = psuwr  • scHwr  • smptypwt  • sesswt  • stuschw 


where  PSUWT,  SCHWT,  SMPTYPWT^  SESSWT^  and  STUSCHW  are,  respectively,  the  reciprocals  of  the 
preceding  probabilities. 

Variations  across  the  various  1996  assessments  in  probabilities  of  selection,  and  consequently  of 
weights,  were  introduced  by  design,  either  to  increase  the  effectiveness  of  the  sample  in  achieving  its  goals 
of  reporting  for  various  subpopulations,  or  to  achieve  increased  efficiency  per  unit  of  cost. 

10.2.1.2  Session  Nonresponse  Adjustment  {SESNRF) 

Sessions  were  assigned  to  schools  before  cooperation  status  was  final.  The  session  nonresponse 
adjustment  was  intended  to  compensate  for  session  type  nonresponse  due  to  refusing  schools  or  individual 
session  types  not  conducted.  The  first  three  digits  of  PSU  stratum,  called  subuniverse  (formed  by  crossing 
the  PSU  major  stratum  and  the  first  socioeconomic  characteristic  used  to  define  the  final  PSU  stratum;  see 
Section  3.2  for  more  detail)  were  used  in  calculating  nonresponse  adjustments.  The  adjustment  factors 
were  computed  separately  within  classes  formed  by  subuniverse  within  sample  type  for  mathematics  and 
science,  and  by  subuniverse  for  the  other  assessment  types.  Occasionally,  additional  collapsing  of  classes 
was  necessary  to  improve  the  stability  of  the  adjustment  factors,  especially  for  the  smaller  assessment 
components.  Most  classes  needing  collapsing  contained  small  numbers  of  cooperating  schools. 
Occasionally,  classes  with  low  response  rates  were  collapsed. 

In  subuniverse  s in  session  type  /i,  the  session  nonresponse  adjustment  factor  SESNRF was  given 


by 


^ PSUWTi  • SCHWTi  • SMPTYPWTi  • SESSWTm  • G, 


SESNRF^  = ^ 


^ PSUWTi  • SCHWTi  • SMPTYPWTi  • SESSWTm  • Gi 


where 


PSUWTi 


the  PSU  weight  for  the  PSU  containing  school  i\ 


SCHWTi 


the  school  weight  for  school  i\ 


SMPTYPWTi 


the  sample  type  weight  for  school  i\ 


SESSWTf^i 


the  session  allocation  weight  for  session  type  h in  school  i\ 


the  estimated  number  of  grade-eligible  students  in  school  i (the  values  of 
Gi  were  based  on  QED  data); 


set 


consists  of  all  in-scope  originally  sampled  schools  allocated  to  session 
type  h in  subuniverse  s (excluding  substitutes);  and 


set  Q, 


consists  of  all  schools  allocated  to  session  type  h in  subuniverse  s that 
ultimately  participated  (including  substitutes). 


It  should  be  noted  that  the  nonresponse  adjustments  assume  that  nonresponse  occurs  at  random 
within  the  categories  within  which  adjustments  are  made  (see  Little  & Rubin,  1987).  Some  degree  of  bias 
could  result  to  the  extent  that  this  assumption  is  false. 


10.2.1.3  School-Level  Poststratification  Adjustment  at  Grade  8 (SCHPSF) 

As  discussed  earlier,  the  weights  at  grade  8 were  poststratified  so  that  the  resulting  estimates  of 
number  of  public  schools  by  percent  minority  enrollment  corresponded  to  totals  based  on  information  from 
the  sampling  frame  of  schools. 

Poststratification  adjustments  were  calculated  separately  by  sample  type  within  assessment  type. 
For  the  descriptions  of  the  information  for  stratification,  see  Section  3.3.  Control  totals  were  calculated  as 
the  total  estimated  number  of  grade-eligible  students  (based  on  QED  data)  in  public  schools,  by  percent 
minority  enrollment  category.  The  percent  minority  enrollment  categories  used  to  form  adjustment  cells 
were  0-4,  5-14,  15-29,  30-36,  37-44, 45-54,  55-79,  80-94,  and  95-100  for  mathematics  and  science  sample 
type  2,  and  0-4,  5-14,  15-29,  30-44, 45-79,  and  80-100  for  all  other  assessment  type/sample  type 
combinations.  The  smaller  sample  sizes  in  the  latter  group  did  not  support  using  more  categories.  The 
poststratification  factor  for  each  class  c and  session  type  h is  computed  by 


SCHPSFhc  = 


TOTALc 

^ PSUWTi  • SCHWTi  • SMPTYPWTi  • SESSWTu  • SESNRFh.,  • Gi 

ieDiic 


where 


TOTAL, 

- 

the  total  number  of  grade-eligible  students  in  class  c,  from  the  sampling 

frame; 

PSUWTi 

- 

the  PSU  weight  for  the  PSU  containing  school  /; 

SCHWTi 

= 

the  school  weight  for  school  /; 

SMPTYPWTi 


the  sample  type  weight  for  school  /; 


SESSWT,^ 


SESNRF„, 

Gi 


Set  Dhc 


the  session  allocation  weight  for  session  type  h in  school  /; 

the  session  nonresponse  adjustment  factor  for  subuniverse  s in  session 
type  h\ 

the  estimated  number  of  grade-eligible  students  in  school  i (the  values  of 
G/  were  based  on  QED  data);  and 

consists  of  the  public  schools  in  class  c that  participated  in  session  type  h. 


For  some  sample  types  in  advanced  mathematics  and  mathematics  estimation,  two  or  more 
poststratification  classes  were  collapsed  into  one  to  improve  the  stability  of  the  adjustment  factors.  Private 
schools  and  new  schools  (those  schools  added  to  the  sample  through  the  new  school  sampling  procedure, 
and  so  not  included  on  the  sampling  frame)  received  school-level  poststratification  factors  of  1.0. 
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10.2.1.4  Student  Nonresponse  Adjustment  (STUNRF) 

Student  nonresponse  adjustment  factors  were  computed  separately  for  each  session  type.  For 
students  in  the  main  samples,  the  adjustment  classes  were  based  on  sample  type  (for  mathematics  and 
science),  subuniverse,  modal  age  status,  and  race  class  (White  or  Asian/Pacific  Islander,  other).  In  some 
cases,  two  or  more  nonresponse  classes  were  collapsed  into  one  to  improve  the  stability  of  the  adjustment 
factors.  For  each  class  c in  session  type  /i,  the  student  nonresponse  adjustment  factor  STUNRF^c  is 
computed  by  . 


PSUWTj  • SCHWTj  • SCHPSFj  • SMPTYPWT.  • SESSWT,j  • SESNRF,j  • STUSCHW,j 

STUNRF^^  ^ 

2-  PSUWTj  • SCHWTj  • SCHPSFj  • SMPTYPWT j • SESSWT,j  • SESNRF,j  • STUSCHW,j 


where 


PSUWTj 


SCHWTj 


SCHPSFj 


SMPTYPWTj  = 


SESSWT^j 

SESNRF^j 

STUSCHW^j 

SetA^^ 

Set 


the  PSU  weight  for  the  PSU  containing  student  J; 

the  school  weight  for  the  school  containing  student  J; 

for  grade  8,  the  school  poststratification  factor  for  the  school  containing 
student  J (set  to  1.0  for  grades  4 and  12); 

the  sample  type  weight  for  the  school  containing  student  J; 

the  session  allocation  weight  for  the  school  containing  student  J in  session 
type  h; 

the  session  nonresponse  adjustment  factor  for  the  school  containing 
student  J in  session  type  h; 

the  within-school  student  weight  for  student  J in  session  type  h; 

consists  of  the  students  in  class  c who  were  sampled  for  session  type  h and 
not  excluded;  and 

consists  of  the  students  in  class  c who  were  assessed  in  session  type  h. 


Excluded  students  received  nonresponse  adjustments  of  1.0. 


10.2.1.5  Trimming  of  Weights 

In  a number  of  cases,  students  were  assigned  relatively  large  weights.  One  cause  of  large  weights 
was  underestimation  of  the  number  of  eligible  students  in  some  schools  leading  to  inappropriately  low 
probabilities  of  selection  for  those  schools.  A second  major  cause  is  the  presence  of  large  schools  (high 
schools  in  particular)  in  PSUs  with  small  selection  probabilities.  In  such  cases,  the  maximum  permissible 
within-school  sampling  rate  (determined  by  the  maximum  sample  size  allowed  per  school — see 
Chapter  3)  could  well  be  smaller  than  the  desired  overall  within-PSU  sampling  rate  for  students.  Large 
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weights  arose  also  because  very  small  schools  were,  by  design,  sampled  with  low  probabilities.  Other 
large  weights  arose  as  the  result  of  high  levels  of  nonresponse  coupled  with  low  to  moderate  probabilities 
of  selection,  and  the  compounding  of  nonresponse  adjustments  at  various  levels. 

Students  with  notably  large  weights  have  an  unusually  large  impact  on  estimates  such  as  weighted 
means.  Since,  under  some  simplifying  assumptions,  the  variability  in  weights  contributes  to  the  variance  of 
an  overall  estimate  by  an  approximate  factor  1 + V^,  where  is  the  relative  variance  of  the  weights,  an 
occasional  unusually  large  weight  is  likely  to  produce  large  sampling  variances  of  the  statistics  of  interest, 
especially  when  the  large  weights  are  associated  with  students  with  atypical  performance  characteristics. 

To  reduce  this  problem,  a procedure  of  trimming  a few  of  the  more  extreme  weights  to  values 
somewhat  closer  to  the  mean  weight  was  applied.  This  trimming  can  increase  the  accuracy  of  the  resulting 
survey  estimates,  substantially  reducing  and  hence  the  sampling  variance,  while  introducing  a small 
bias.  The  trimming  algorithm  was  identical  to  that  used  since  1984,  and  had  the  effect  of  trimming  the 
weights  of  students  from  any  school  that  contributed  more  than  a specified  proportion,  to  the  estimated 
variance  of  the  estimated  number  of  students  eligible  for  assessment.  The  trimming  was  done  separately 
within  sample  type  for  mathematics  and  science,  and  overall  for  mathematics  estimation  and  mathematics 
theme.  Trimming  was  not  done  for  the  advanced  mathematics  or  advanced  science  assessments  because 
advanced  students  were  expected  to  be  concentrated  in  certain  schools,  so  that  the  trimming  algorithm  was 
not  appropriate  in  these  cases.  In  each  case,  the  value  of  the  proportion  ^ was  chosen  to  be  10/ AT,  where  K 
was  the  number  of  schools  in  which  a specified  assessment  was  conducted.  The  number  of  schools  where 
weights  were  trimmed  was  no  more  than  seven  in  any  one  assessment.  The  most  extreme  trimming  factors 
applied  were  of  the  order  of  0.65;  trimming  affects  the  weights  of  only  a very  small  proportion  of  the 
assessed  and  excluded  students. 

Table  10-2  shows  the  distributions  of  eligible  students  based  on  the  trimmed  weights  of  assessed 
students  for  the  science  samples  in  sample  type  2 (the  reporting  population)  for  each  grade.  The 
distributions  are  similar  to  those  before  trimming  shown  in  Tables  10-6,  10-7,  and  10-8.  To  the  extent  that 
the  characteristics  in  the  table  are  related  to  student  performance  on  the  science  assessment,  there  is  a 
small  bias  introduced  in  the  assessment  by  trimming. 
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Table  10-2 

Distribution  of  Populations  of  Eligible  Students  Based  on  Trimmed  Weights  of  Assessed  Students 
in  Participating  Schools,  1996  Main  NAEP  Science  Sample,  Sample  Type  2 


Population 

Grade  4 

Grade  8 

Grade  12 

Total  population 

3,389,669 

3,365,499 

2,491,555 

Age  category 

At  modal  age  or  younger 

64.6% 

57.9% 

64.5% 

Older  than  modal  age 

35.4% 

42.1% 

35.5% 

Race/ethnicity  category 

White 

61.0% 

64.2% 

69.1% 

Black 

14.3% 

15.7% 

12.3% 

Hispanic 

16.9% 

13.6% 

11.1% 

Other 

7.8% 

6.6% 

7.5% 

Gender 

Male 

49.6% 

50.2% 

48.3% 

Female 

50.2% 

49.6% 

51.7% 

SD 

Yes 

5.2% 

5.7% 

3.1% 

No 

94.8% 

94.3% 

96.9% 

LEP 

Yes 

3.0% 

1.9% 

2.2% 

No 

97.0% 

98.1% 

97.8% 

SD,  LEP 

SD  yes,  LEP  yes 

0.1% 

0.1% 

0.0% 

SD  yes,  LEP  no 

5.1% 

5.6% 

3.0% 

SD  no,  LEP  yes 

3.0% 

1.8% 

2.1% 

SD  no,  LEP  no 

91.8% 

92.5% 

94.8% 

‘ For  a very  small  percentage  of  students  at  grades  4 and  8,  gender  is  unknown. 


10.2.1.6  Reporting  Factor 

Each  set  of  trimmed  weights  for  a given  sample  type  sums  to  the  target  population.  Reporting 
factors  were  assigned  to  students  in  order  to  scale  back  the  trimmed  weights  so  that  final  student 
(reporting)  weights  within  each  reporting  population  (which  may  combine  students  from  different  sample 
types)  sum  to  the  target  population.  The  reporting  factors  assigned  to  students  are  specific  to  the  reporting 
populations  defined  in  Table  10-1.  Each  assessed  and  excluded  student  in  the  reporting  population 
received  a reporting  factor  as  shown  in  Table  10-3  on  the  following  page. 
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Table  10-3 

Reporting  Factors  for  Assessed  and  Excluded  Students 


Non-SD/LEP  Students 

SD/LEP  Students 

1 

1 

2 

3 

1 

2 3 

Grade  4 

Mathematics 

0.5 

0.5 

— 

1 

— — 

Science 

— 

1 

— 

— 

1 — 

Mathematics  theme 

— 

1 

— 

— 

1 — 

Mathematics  estimation 

1 

— 

— 

1 

— — 

Grade  8 

Mathematics 

0.5 

0.5 

— 

1 

— — 

Science 

— 

1 

— 

— 

1 — 

Mathematics  theme 

— 

1 

— 

— 

1 — 

Mathematics  estimation 

1 

— 

— 

1 

— — 

Advanced  mathematics  (B) 

— 

0.35 

0.65 

— 

1 — 

Grade  12 

Mathematics 

0.5 

0.5 

— 

1 

— — 

Science 

— 

1 

— 

— 

1 — 

Mathematics  theme 

— 

1 

— 

— 

1 — 

Mathematics  estimation 

0.6667 

0.3333 

— 

1 

— — . 

Advanced  mathematics  (B) 

— 

— 

1 

— 

— 1 

Advanced  science  (C) 

— 

1 

— 

— 

1 — 

10.2.1.7  Student-Level  Poststratification 


As  in  most  sample  surveys,  the  respondent  weights  are  random  variables  that  are  subject  to 
sampling  variability.  Even  if  there  were  no  nonresponse,  the  respondent  weights  would  at  best  provide 
unbiased  estimates  of  the  various  subgroup  proportions.  However,  since  unbiasedness  refers  to  average 
performance  over  a conceptually  infinite  number  of  replications  of  the  sampling,  it  is  unlikely  that  any 
given  estimate,  based  on  the  achieved  sample,  will  exactly  equal  the  population  value.  Furthermore,  the 
respondent  weights  have  been  adjusted  for  nonresponse  and  a few  extreme  weights  have  been  reduced  in 
size. 


To  reduce  the  mean  squared  error  of  estimates  using  the  sampling  weights,  these  weights  were 
further  adjusted  so  that  estimated  population  totals  for  a number  of  specified  subgroups  of  the  population, 
based  on  the  sum  of  weights  of  students  of  the  specified  type,  were  the  same  as  presumably  better 
estimates  based  on  composites  of  estimates  from  the  1993  and  1994  Current  Population  Survey  and  1996 
population  projections  made  by  the  U.S.  Census  Bureau.  This  adjustment,  called  poststratification,  is 
intended  especially  to  reduce  the  mean  squared  error  of  estimates  relating  to  student  populations  that  span 
several  subgroups  of  the  population,  and  thus  also  to  reduce  the  variance  of  measures  of  changes  over  time 
for  such  student  populations. 


The  poststratification  in  1996  was  done  for  the  mathematics,  science,  mathematics  estimation,  and 
mathematics  theme  assessments  in  each  grade.  The  advanced  mathematics  and  advanced  science 
assessments  were  not  poststratified.  Within  each  grade  and  assessment  type  group,  poststratification 
adjustment  cells  were  defined  in  terms  of  race,  ethnicity,  and  Census  region  as  shown  in  Table  10-4.  Note 
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that  NAEP  region  was  used  in  previous  years  instead  of  Census  region.  This  change  was  made  because  the 
data  from  the  Current  Population  Survey  and  Census  Projections  are  more  reliable  for  Census  regions  than 
for  NAEP  regions. 


Table  10-4 

Major  Subgroups  for  Poststratification  in  1996 


Subgroup 

Race 

Ethnicity 

Census  Region' 

1 

White 

Not  Hispanic 

Northeast 

2 

White 

Not  Hispanic 

Midwest 

3 

White 

Not  Hispanic 

South 

4 

White 

Not  Hispanic 

West 

5 

Any 

Hispanic 

Any 

6 

Black 

Not  Hispanic 

Any 

7 

Other 

Not  Hispanic 

Any 

' Census  region  is  the  same  as  the  NAEP  region  in  Table  3-2  used  for  stratification  and 
reporting,  except  that  DE,  DC,  and  MD  moved  from  the  Northeast  to  the  South,  OK  and 
TX  moved  from  the  West  to  the  South,  Central  is  called  Midwest,  and  Southeast  is  called 
South. 

These  subgroups  were  used  as  adjustment  cells  at  grade  12.  Each  of  these  cells  accounts  for 
between  5 percent  (Subgroup  7)  and  21  percent  (Subgroup  3)  of  the  population.  For  grades  4 and  8,  each  of 
the  seven  subgroups  was  further  divided  into  two  eligibility  classes:  of  modal  age  and  not  of  modal  age. 

The  procedure  used  at  grade  12  was  adopted  because  the  independent  estimates  of  the  numbers  of 
students  in  the  population  did  not  provide  consistent  data  on  the  numbers  of  twelfth  grade  students  by  age. 
Specifically,  the  counts  of  twelfth  grade  students  age  17  and  older  are  not  reliable  because  they  include 
adult  education  students.  This  procedure  has  been  used  since  1988.  (See  Rust,  Bethel,  Burke,  & Hansen, 
1990,  and  Rust,  Burke,  & Fahimi,  1992,  for  further  details.) 

Thus,  there  were  7 or  14  cells  for  poststratification.  The  poststratified  weight  for  each  student 
within  a particular  cell  was  the  student’s  base  weight,  with  adjustments  for  nonresponse  and  trimming,  and 
the  reporting  factor  from  Section  10.2.1.6,  times  a poststratification  factor.  The  poststratification  factor  for 
studenty  in  session  type  h and  poststratification  adjustment  class  c is  given  by 


TOTAL 


where 


^ ^ he 

1 

M 

• SESNRFj  • SCHPSFj  • STUNRFj  • TRIMFCTRj  • RPTFCTRj 

yeCj,. 

Wsj 

= 

the  base  weight  for  student  j (see  Section  10.2.1.1); 

TOTAL, 

— 

the  total  number  of  grade-eligible  students  in  class  c,  from  the  October 
1993  and  1994  Current  Population  Surveys  and  1996  population 
projections; 

SESNRFj 

the  session  nonresponse  adjustment  factor  for  the  school  containing 
student  y in  session  type  h\ 
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SCHPSFj 

for  grade  8,  the  school  poststratification  factor  for  the  school  containing 
student  j (set  to  one  for  grades  4 and  12); 

STUNRFj 

the  student  nonresponse  adjustment  for  student  j; 

TRIMFCTRj  = 

the  trinuning  factor  for  student  j', 

RPTFCTRj 

the  reporting  factor  for  student y; 

Set  Che 

consists  of  the  students  in  class  c who  were  assessed  in  session  type  h, 
except  those  at  grade  12  who  were  age  17  or  older. 

Note  that  students  at  grade  12  who  were  age  17  or  older  received  the  poststratification  factor  according  to 
their  adjustment  class  and  session  type  even  though  they  were  not  used  in  calculating  the  factor.  Details  of 
the  procedures  used  to  obtain  totals  in  the  numerator  of  the  adjustment  factor  are  provided  in  Wallace  and 
Rust  (1999). 


10.2.1.8  The  Final  Student  Reporting  Weights 

The  final  weight  assigned  to  a student  is  the  student  full-sample  reporting  weight.  This  weight  is 
the  student’s  base  weight  after  the  application  of  the  various  adjustments  described  above.  The  student 
full-sample  reporting  weight  was  used  to  derive  all  estimates  of  population  and  subpopulation 
characteristics  that  have  been  presented  in  the  various  NAEP  reports,  including  simple  estimates  such  as 
the  proportion  of  students  of  a specified  type  who  would  respond  in  a certain  way  to  an  item  and  more 
complex  estimates  such  as  tnein  proficiency  levels.  The  distributions  of  the  final  student  reporting  weights 
are  given  in  Table  10-5. 

As  indicated  earlier,  under  some  simplifying  assumptions  the  factor  1 -i-  indicates  the 
approximate  relative  increase  in  variance  of  estimates  resulting  from  the  variability  in  the  \yeights.  The 
factor  for  each  sample  is  readily  derivable  from  Table  10-5  by  squaring  the  ratio  of  the  standard 
deviation  to  the  mean  weight.  These  factors,  resulting  from  the  combined  effect  of  the  variations  in 
weights  introduced  by  design  and  from  other  causes,  are  discussed  in  Section  10.2.3. 


10.2.2  Evaluation  of  Potential  for  Bias  Resulting  from  School  and  Student  Nonparticipation 

Although  school  and  student  nonresponse  adjustments  are  intended  to  reduce  the  potential  for 
nonparticipation  to  bias  the  assessment  results,  they  cannot  completely  eliminate  this  potential  bias  with 
certainty.  The  extent  of  bias  remains  unknown,  of  course,  since  there  are  no  assessment  data  for  the 
nonparticipating  schools  and  students. 

Some  insight  can  be  gained  about  the  potential  for  residual  nonresponse  bias,  however,  by 
examining  the  weighted  school-  and  student-level  distributions  of  characteristics  known  for  both 
participants  and  nonparticipants,  especially  for  those  characteristics  known  or  thought  likely  to  be  related 
to  achievement  on  the  assessment.  If  the  distributions  for  the  full  sample  of  schools  (or  students)  without 
the  use  of  nonresponse  adjustments  are  close  to  those  for  the  participants  with  nonresponse  adjustments 
applied,  there  is  reason  to  be  confident  that  the  bias  from  nonparticipation  is  small. 
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Table  10-5 

Distribution  of  Final  Student  Reporting  Weights,  1996  Main  Samples 

Number  Standard  25th  75th 

Sample of  Cases  Mean  Deviation  Minimum  Percentile  Median  Percentile 
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There  are  several  school-level  characteristics  available  for  both  participating  and  nonparticipating 
schools.  The  tables  below  show  the  combined  impact  of  nonresponse  and  of  the  nonresponse  adjustments 
on  the  distributions  of  schools  (weighted  by  the  estimated  number  of  eligible  students  enrolled)  and 
students,  by  the  type  of  school  (public.  Catholic,  other  nonpublic)  the  size  of  the  school  as  measured  by  the 
estimated  number  of  eligible  students  enrolled,  and  the  urban/rural  nature  of  the  place  where  the  school  is 
located.  Three  size  classes  have  been  defined  for  each  grade.  The  data  are  for  the  science  assessment, 
sample  type  2 (the  reporting  population).  Science  from  sample  type  2 was  chosen  because  it  is  the  largest 
assessment  at  each  grade.  It  is  assumed  that  other  large  assessments  would  behave  similarly. 

Several  student-level  characteristics  are  available  for  both  absent  and  assessed  students.  The  tables 
that  follow  show  the  impact  of  school  nonresponse  and  nonresponse  adjustments,  and  student  nonresponse 
and  nonresponse  adjustments  on  the  distributions  of  eligible  students  for  each  grade.  This  discussion  also 
focuses  on  the  science  assessment  for  sample  type  2,  since  it  is  the  largest.  The  distributions  are  presented 
by  age  category  (at  or  below  modal  age,  and  above  modal  age),  race  category  (White,  Black,  Hispanic,  and 
other),  gender,  SD,  and  LEP. 

Table  10-6  shows  the  weighted  marginal  distributions  of  students  for  each  of  the  three 
classification  variables  for  each  grade,  using  weighted  eligible  schools.  The  distributions  before  school 
nonresponse  adjustments  are  based  on  the  full  sample  of  in-scope  schools  for  science— those  participating, 
plus  those  refusals  for  which  no  substitute  participated.  The  distributions  after  school  nonresponse 
adjustments  are  based  only  on  participating  schools  for  science,  with  school  nonresponse  adjustments 
applied  to  them. 

It  can  be  seen  from  the  tables  that,  even  though  the  level  of  school  nonparticipation  is  as  high  as 
22.6  percent  for  grade  12  (see  Table  3-1 1)  and  somewhat  lower  for  the  other  grades,  for  the  most  part,  the 
distributions  for  the  three  characteristics  considered  remain  similar.  Exceptions  may  be  medium  and  large 
schools,  and  midsize  cities  and  urban  fringe  of  large  cities  at  grade  12;  and  urban  fringe  of  large  cities  and 
rural  nonMSAs  at  grade  4. 
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Table  10-6 

Distribution  of  Populations  of  Eligible  Students  Based  on  Full  Weighted  Sample  of  Eligible  Schools, 
Before  and  After  School  Nonresponse  Adjustments,  1996  Main  NAEP  Science  Samples,  Sample  Type  2 


Population 

Grade  4 

Before  After 

Grade  8 

Before  After 

Grade  12 

Before  After 

Total  population 

3,777,554 

3,777,554 

3,198,390 

3,198,390 

2,762,448 

2,762,448 

School  type 

Catholic 

5.7% 

7.2% 

6.0% 

6.8% 

5.5% 

5.9% 

Other  Nonpublic 

5.6% 

4.3% 

4.7% 

4.6% 

4.2% 

3.4% 

Public 

88.7% 

88.5% 

89.3% 

88.7% 

90.3% 

90.7% 

School  size* 

1 

18.8% 

19.8% 

10.8% 

11.6% 

5.0% 

4.7% 

2 

46.4% 

47.2% 

55.7% 

55.8% 

70.5% 

67.0% 

3 

34.8% 

33.0% 

33.5% 

32.6% 

24.5% 

28.3% 

School  location 

Large  city 

18.3% 

19.9% 

17.7% 

17.9% 

15.4% 

16.8% 

Midsize  city 

19.4% 

19.7% 

18.1% 

18.1% 

16.3% 

18.3% 

Urban  fringe/large  city 

23.9% 

24.0% 

23.1% 

21.8% 

23.3% 

21.2% 

Urban  fringe/midsize  city 

13.7% 

11.9% 

15.8% 

16.7% 

15.3% 

14.4% 

Large  town 

0.3% 

0.5% 

0.7% 

0.7% 

1.5% 

0.9% 

Small  town 

10.5% 

8.8% 

9.9% 

8.8% 

14.4% 

15.5% 

Rural  MSA 

2.3% 

1.8% 

5.1% 

6.1% 

3.7% 

3.7% 

Rural  nonMSA 

11.5% 

13.5% 

9.6% 

9.9% 

10.2% 

9.5% 

' Distributions  by  school  size  are  not  comparable  to  previous  assessments,  since  students  were  eligible  by  grade  only  (instead  of  by  grade 
or  age)  in  1996.  School  size  = number  of  eligible  students  enrolled: 


1 

2 

3 

Grade  4 

1-49 

50-99 

100  + 

Grade  8 

1-49 

50-299 

300  + 

Grade  12 

1-49 

50-399 

400  + 

Table  10-7  shows  the  distributions  of  the  same  three  classification  variables,  plus  additional 
distributions  of  student-level  characteristics,  using  weighted  eligible  students.  The  distributions  before 
student  nonresponse  adjustments  are  based  on  assessed  and  absent  science  students  (with  base  weights 
adjusted  for  school  nonparticipation).  The  distributions  after  student  nonresponse  adjustments  are  based  on 
assessed  science  students  only,  with  the  student  nonresponse  adjustments  also  applied  to  them. 
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Table  10-7 

Distribution  of  Populations  of  Eligible  Students  Before  and  After  Student  Nonresponse  Adjustments 

1996  Main  NAEP  Science  Samples,  Sample  Type  2 


Population 

Grade  4 

Before  After 

Grades 

Before  After 

Grade  12 

Before  After 

Total  population 

3,419,493 

3,419,493 

3,428,867 

3,428,867 

2,496,241 

2,496,241 

School  type 

Catholic 

7.7% 

7.8% 

6.1% 

6.3% 

6.4% 

8.1% 

Other  Nonpublic 

4.5% 

4.6% 

4.3% 

4.4% 

3.6% 

4.2% 

Public 

87.8% 

87.6% 

89.6% 

89.3% 

90.0% 

87.6% 

School  location 

Large  city 

18.9% 

18.9% 

13.2% 

13.2% 

17.0% 

16.9% 

Midsize  city 

19.2% 

19.3% 

15.9% 

15.9% 

18.2% 

16.9% 

Urban  fringe/large  city 

25.2% 

25.2% 

20.8% 

20.8% 

21.4% 

21.7% 

Urban  fringe/nrddsize  city 

12.1% 

12.0% 

19.3% 

19.4% 

13.6% 

14.8% 

Large  town 

0.5% 

0.5% 

0.5% 

0.5% 

1.0% 

1.0% 

Small  town 

8.9% 

9.0% 

11.6% 

11.5% 

15.5% 

15.2% 

Rural  MSA 

1.8% 

1.8% 

7.1% 

7.1% 

3.9% 

4.1% 

Rural  nonMSA 

13.3% 

13.3% 

11.6% 

11.7% 

9.4% 

9.6% 

Age  category 

At  modal  age  or  younger 

64.6% 

64.5% 

57.5% 

57.6% 

64.3% 

64.5% 

Older  than  modal  age 

35.4% 

35.5% 

42.5% 

42.4% 

35.7% 

35.5% 

Race/ethnicity  category 

White 

61.4% 

61.2% 

64.9% 

64.7% 

69.4% 

69.1% 

Black 

14.4% 

14.2% 

15.7% 

15.4% 

12^6% 

12.2% 

Hispanic 

16.6% 

16.8% 

13.0% 

13.3% 

11.0% 

11.1% 

Other 

7.6% 

7.8% 

6.4% 

6.5% 

7.1% 

7.5% 

Gender' 

Male 

49.6% 

49.6% 

50.3% 

49.6% 

48.7% 

48.4% 

Female 

50.2% 

50.2% 

49.4% 

50.1% 

51.2% 

51.6% 

SD 

Yes 

5.3% 

5.3% 

6.1% 

5.8% 

3.4% 

3.0% 

No 

94.7% 

94.7% 

93.9% 

94.2% 

96.6% 

97.0% 

LEP 

Yes 

3.0% 

3.0% 

1.9% 

1.9% 

2.1% 

2.2% 

No 

97.0% 

97.0% 

98.1% 

98.1% 

97.9% 

97.8% 

SD,  LEP 

SD  yes,  LEP  yes 

0.1% 

0.1% 

0.1% 

0.1%  • 

0.1% 

0.0% 

SD  yes,  LEP  no 

5.2% 

5.2% 

6.0% 

5.7% 

3.3% 

3.0% 

SD  no,  LEP  yes 

2.9% 

2.9% 

1.8% 

1.8% 

2.1% 

2.1% 

SD  no,  LEP  no 

91.8% 

91.8% 

92.1% 

92.5% 

94.6% 

94.8% 

' Gender  is  unknown  for  a small  percentage  of  students. 


The  rates  of  student  nonparticipation  were  5.1  percent  for  grade  4,  6.9  percent  for  grade  8,  and 
22.5  percent  for  ^ade  12  (see  Table  3-11).  The  table  shows  that  with  one  exception  at  grade  12,  for  the 
distributions  of  type  of  school  attended  and  place  where  the  school  is  located,  the  combined  effect  of 
student  nonparticipation  and  the  subsequent  nonresponse  adjustments  have  resulted  in  very  little  change  in 
distribution.  The  changes  in  the  distribution  of  school  type  at  grade  12  reflect  the  relatively  high 
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nonresponse  rate  of  grade  12  public  school  students  (22.7  percent  versus  8.5  percent  for  nonpublic  school 
students;  see  Table  3-9). 

Table  10-8  shows  the  weighted  distributions  of  eligible  students  in  participating  schools,  using  the 
base  weights  of  assessed  and  absent  students  unadjusted  for  school-level  nonresponse.  Tables  10-7  and  10- 
8 show  that  both  school  and  student-level  nonresponse  and  nonresponse  adjustments  have  little  effect  on 
the  distributions  of  eligible  students  by  age,  race/ethnicity,  gender,  SD  and  LEP.  All  of  the  distributions  in 
the  tables  are  similar. 


Table  10-8 

Distribution  of  Populations  of  Eligible  Students  Before  School  and  Student  Nonresponse  Adjustments 

1996  Main  NAEP  Science  Samples,  Sample  Type  2 


Population 

Grade  4 

Grade  8 

Grade  12 

Total  population 

2,635,218 

2,743,713 

1,935,174 

Age  category 

At  modal  age  or  younger 

64.3% 

56.3% 

63.9% 

Older  than  modal  age 

35.7% 

43.7%  , 

36.1% 

Race/ethnicity  category 

White 

60.8% 

66.6% 

70.3% 

Black 

14.6% 

14.0% 

12.4% 

Hispanic 

16.9% 

13.0% 

10.3% 

Other 

7.7% 

6.5% 

6.9% 

Gender 

Male 

49.7% 

50.3% 

48.8% 

Female 

SD 

50.1% 

49.5% 

51.2% 

Yes 

5.3% 

6.2% 

3.4% 

No 

94.7% 

93.8% 

96.6% 

LEP 

Yes 

3.2% 

2.0% 

2.0% 

No 

96.8% 

98.8% 

98.0% 

SD,  LEP 

SD  yes,  LEP  yes 

0.1% 

0.1% 

0.1% 

SD  yes,  LEP  no 

5.2% 

6.1% 

3.3% 

SD  no,  LEP  yes 

3.1% 

1.9% 

1.9% 

SD  no,  LEP  no 

91.7% 

91.9% 

94.7% 

* Gender  is  unknown  for  a small  percentage  of  students. 


When  comparing  the  distributions  in  Table  10-7  before  and  after  student  nonresponse  adjustments, 
we  expect  the  distributions  by  age  category  and  race/ethnicity  to  be  similar  because  these  variables  were 
used  to  determine  student  nonresponse  adjustment  classes.  However,  the  distributions  by  gender,  SD,  and 
LEP  are  also  similar.  To  the  extent  that  nonrespondents  would  perform  like  respondents  with  the  same 
characteristics  (defined  by  the  classification  variables  in  the  tables),  the  bias  in  the  assessment  data  is 
small. 


Further  information  about  potential  nonresponse  bias  can  be  gained  by  studying  the  absent 
students.  NAEP  proficiency  estimates  are  biased  to  the  extent  that  assessed  and  absent  students  within  the 


0‘-4 

ERIC 


238 


same  weighting  class  differ  in  their  distribution  of  proficiency.  It  seems  likely  that  the  assumption  that 
absent  students  are  similar  in  proficiency  to  assessed  students  is  reasonable  for  some  absent 
students — namely,  those  whose  absence  can  be  characterized  as  random.  Conversely,  it  seems  likely  that 
students  with  longer  and  more  consistent  patterns  of  absenteeism — such  as  truants,  dropouts,  near 
dropouts,  and  the  chronically  ill— are  unlikely  to  be  as  proficient  as  their  assessed  counterparts. 

In  the  1996  assessments,  schools  were  asked  to  classify  each  absent  student  into  one  of  nine 
categories.  The  results  of  this  classification  for  the  science  assessment  are  shown  in  Table  10-9.  The 
discussion  focuses  on  the  science  assessment  because  it  is  the  largest.  It  is  assumed  that  the  other  large 
assessments  would  behave  similarly. 

Table  10-9  shows  that,  as  anticipated,  the  majority  of  absence  from  the  assessment  was  the  result 
of  an  absence  from  school  of  a temporary  and  unscheduled  nature.  The  table  shows  that  absence  among 
twelfth  graders  occurs  at  about  four  times  the  rate  of  absence  among  fourth  or  eighth  graders.  The 
proportion  of  absence  classified  as  temporary  differs  somewhat  by  grade,  but  is  of  the  same  magnitude  for 
grades  8 and  12.  These  two  facts  taken  together  suggest  strongly  that  a substantial  proportion  of  the 
temporary  absences  among  twelfth  grade  students  is  not  a result  of  illness,  because  such  absences  are 
occurring  at  almost  three  times  the  rate  that  they  do  among  fourth  or  eighth  grade  students.  Whereas  it 
might  be  reasonable  to  regard  temporary  absence  due  to  illness  as  independent  of  proficiency,  for  other 
temporary  absences,  this  appears  less  tenable.  The  data  in  the  table  give  support  to  the  contention  that,  at 
grade  4,  student  absences  are  unlikely  to  introduce  any  significant  bias  into  NAEP  estimates.  The  absentee 
rate  is  low;  most  absences  are  temporary,  and  three  quarters  of  the  remaining  absences  are  a result  of 
parental  refusal. 


Table  10-9 

Weighted  Distribution  of  Absent  Students  by  Nature  of  Absenteeism  for  All  Grades 
1996  Science  Assessment,  Sample  Type  2 


Nature  of  Absenteeism 

Grade  4 

Grade  8 

Grade  12 

Temporary  absence' 

78.2% 

68.0% 

63.6% 

Long-term  absence^ 

1.6% 

1.4% 

0.8% 

Chronic  truant 

0.2% 

2.7% 

1.4% 

Suspended  or  expelled 

0.7% 

6.4% 

0.4% 

Parent  refusal 

16.4% 

13.1% 

9.2% 

Student  refusal 

0.0% 

3.2% 

12.0% 

In  school,  did  not  attend  session 

0.0% 

3.0% 

7.0% 

In  school,  not  invited^ 

0.0% 

0.8% 

0.0% 

Other 

0.3% 

0.9% 

3.9% 

Missing 

2.6% 

0.4% 

1.8% 

Total  absentee  sample 

384 

569 

2,269 

Total  sample  size 

7,689 

8,343 

9,807 

Overall  absentee  rate 

5.0% 

6.8% 

23.1% 

* Absent  less  than  two  weeks  due  to  illness,  disability,  or  excused  absence. 

^ Absent  more  than  two  weeks  due  to  illness  or  disability. 

^ In  school,  but  not  invited  to  assessment  session  due  to  disruptive  behavior. 


At  grades  8 and  12,  however,  a significant  component  of  absenteeism  is  not  temporary  or  due  to 
parental  refusal.  Chronic  truants,  those  suspended,  and  those  in  school  but  not  invited,  constitute  the 
obvious  candidates  for  potential  bias.  These  groups  comprise  12.1  percent  of  absent  students  at  grade  8 (or 
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0.8  percent  of  the  total  sample)  and  8.8  percent  of  absent  students  at  grade  12  (or  2.0  percent  of  the  total 
sample).  Thus  their  potential  for  introducing  significant  bias  under  the  current  procedures  is  minor. 


10.2.3  Derivation  of  Modular  Weights  for  the  Main  Samples 

As  discussed  earlier,  modular  weights  were  computed  to  facilitate  analyses  involving  students 
from  a single  sample  type.  The  same  procedures  were  used  to  derive  modular  and  reporting  weights  up 
through  the  weight  trimming  step  described  in  Section  10.2.1.5.  After  trimming,  weighting  continued  in 
two  parallel  processes.  Final  student  reporting  weights  were  the  result  of  one  of  these  processes,  and 
modular  weights  were  the  result  of  the  other. 

Modular  weights  differ  from  reporting  weights  in  two  ways.  First,  they  do  not  contain  the 
reporting  factor  described  in  Section  10.2.1.6.  The  second  difference  lies  in  the  manner  in  which  the 
weights  were  poststratified.  The  modular  weights  were  poststratified  as  described  in  Section  10.2.1.7, 
except  that  each  sample  type  within  each  grade  and  session  was  poststratified  separately.  The  same  initial 
adjustment  cells  were  used:  7 cells  based  on  race/region  for  each  session/sample  type  combination  at  grade 
12,  and  14  cells  based  on  race/region  and  eligibility  class  (of  modal  age,  not  of  modal  age)  for  each 
session/  sample  type  combination  at  grades  4 and  8.  Some  adjustment  factors  are  quite  variable  for  the 
same  adjustment  cell  across  different  sample  types  for  the  same  grade  and  session.  This  indicates  that  the 
individual  samples  by  sample  type  may  not  be  particularly  stable,  especially  for  the  smaller  sessions  of 
mathematics  theme  and  mathematics  estimation. 

The  modular  weight  is  the  student’s  base  weight  after  the  application  of  the  various  adjustments 
described  in  Section  10.2.1,  except  for  the  reporting  factor,  and  with  the  new  poststratification  factor 
described  above.  The  distributions  of  the  modular  weights  are  given  in  Table  10-10.  Note  that  modular 
weights  are  identical  to  reporting  weights  for  a particular  grade/session/sample  type  combination  when  that 
sample  type  is  the  only  one  included  in  the  reporting  population  for  that  grade  and  session. 


10.2.4  Derivation  of  Student  Weights  for  the  Long-Term  Trend  Samples 


Final  student  weights  were  derived  for  the  long-term  trend  samples  in  a manner  similar  to  that  used 
in  1994.  The  procedure  was  identical  to  that  used  to  derive  reporting  weights,  described  in  Section  10.2.1, 
except  as  noted  below. 

As  in  1994,  excluded  students  for  all  subjects  in  the  long-term  trend  samples  were  weighted 
together,  separately  from  assessed  students. 

Base  weight.  A student’s  base  weight  is  the  reciprocal  of  the  product  of  four  factors — all  of  the 
factors  used  for  reporting  weights  except  sample  type  weight  {SMPTYPWT). 

Session  nonresponse  adjustments.  Session  nonresponse  adjustments  were  calculated  separately  at 
each  age  class  for  the  spiral  assessment,  the  tape  assessment,  and  excluded  students,  within  classes  formed 
by  subuniverse.  The  formula  for  the  adjustment  does  not  contain  sample  type  weight  {SMPTYPWT),  and 
for  excluded  students,  it  does  not  contain  session  allocation  weight  {SESSWT).  G,  is  the  estimated  number 
of  age-  plus  grade-eligible  students  in  the  school  for  the  spiral  assessment  and  excluded  students,  and  the 
estimated  number  of  age-eligible  students  for  the  tape  assessment.  For  excluded  students.  Sets  B and  C are 
not  specific  to  any  particular  session. 
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School-level poststratification.  There  was  no  school-level  poststratification  for  the  long-term  trend 
assessments. 

Student  nonresponse  adjustments.  Student  nonresponse  adjustments  were  calculated  separately  at 
each  age  class  for  the  spiral  assessment  and  the  tape  assessment  within  classes  formed  by  subuniverse  and 
modal  grade  status  (at  or  above  modal  grade,  below  modal  grade).  For  excluded  students  at  each  age  class, 
the  adjustments  were  calculated  within  classes  formed  by  subuniverse.  The  formula  for  the  adjustment  does 
not  contain  sample  type  weight  (SMPTYPWT)  or  the  school-level  poststratification  factor  (SCHPSF),  and  for 
excluded  students,  it  does  not  contain  session  allocation  weight  (SESSWT).  For  excluded  students.  Set  A 
consists  of  all  excluded  students  in  class  C,  and  Set  B consists  of  the  excluded  students  in  class  C for  whom 
an  excluded  student  questionnaire  was  completed. 

Trimming.  Trimming  was  done  separately  for  the  spiral  assessment,  the  tape  assessment,  and 
excluded  students  at  each  age  class. 

Reporting  factor:  There  was  no  reporting  factor  for  the  long-term  trend  assessments. 

Student-level  poststratification.  Poststratification  adjustments  were  calculated  separately  at  each 
age  class  for  the  spiral  assessment,  the  tape  assessment,  and  excluded  students.  Adjustment  cells  were 
formed  by  race/region  (as  described  in  Section  10.2.1.7)  and  eligibility  class  (eligible  by  grade  and  of  modal 
age,  eligible  by  age  only,  and  eligible  by  grade  but  not  of  modal  age).  Thus  21  cells  were  used  for  the  spiral 
assessment  and  excluded  students  at  each  age  class.  Seven  cells  (by  race/region  only)  were  used  for  the  tape 
assessment  at  each  age  class.  For  each  cell  the  poststratification  factor  is  a ratio  whose  denominator  is  the 
sum  of  weights  (after  adjustments  for  nonresponse  and  trimming)  of  assessed  and  excluded  students,  and 
whose  numerator  is  an  adjusted  estimate  of  the  total  number  of  students  in  the  population  who  are  members 
of  the  cell. 

Final  student  weights.  The  final  weight  assigned  to  each  student  is  the  student’s  base  weight  after 
application  of  the  various  adjustments  described  above.  The  distributions  of  the  final  student  weights  for  the 
long-term  trend  samples  are  given  in  Table  10-11. 


10.2.5  Other  Weights 

Special  weighting  adjustments  were  developed  for  certain  subsets  of  the  fourth-grade  and  eighth- 
grade  students  assessed  in  the  main  samples.  The  weights  for  these  subsets,  with  these  adjustments  applied, 
were  used  in  equating  the  results  of  the  national  and  state  assessments  for  subjects  they  had  in  common. 
Also,  weights  appropriate  for  analyzing  school-level  data  files  were  developed. 


10.2.5.1  Weights  for  Equating  National  and  State-by-State  Samples 

Weights  for  Equating  National  and  State-by-State  Assessments.  The  fourth-grade  mathematics  and 
eighth-grade  mathematics  and  science  assessments  conducted  in  February  1996  in  the  NAEP  1996  State 
Assessment  consisted  of  identical  assessment  material  to  that  administered  in  the  corresponding  national 
main  sample  sessions.  Technical  details  of  the  NAEP  1996  State  Assessments  are  given  in  Allen,  Jenkins, 
Kulick,  and  Zelenak  (1997)  and  Allen,  Swinton,  Isham,  and  Zelenak  (1998).  The  national  and  state-by-state 
assessments  were  equated  so  that  state  and  national  results  could  be  reported  on  a common  scale.  The 
equating  was  achieved  by  using  from  each  assessment  that  part  of  the  sample  representing  a common 
population.  For  the  national  samples,  this  consisted  of  those  fourth-grade  or  eighth-grade  public-school 
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students  from  a participating  state  (including  the  District  of  Columbia)  who  were  assessed  in  the  main 
sample  mathematics  dr  (for  grade  8)  science  assessment  reporting  samples. 


Although  each  sample  of  students  received  appropriate  weights  from  the  weighting  procedure  used 
for  the  national  assessment,  in  an  effort  to  increase  the  precision  of  the  equating  process,  an  additional 
weighting  adjustment  was  developed  and  applied  to  each  subsample  by  grade  and  subject,  solely  for  use  in 
equating.  For  each  subsample,  the  distributions  of  the  main  sample  reporting  weights  for  three  categorical 
variables  were  adjusted  to  agree  closely  with  those  obtained  from  the  weighted  aggregate  sample  from  the 
state  assessments  from  the  participating  states.  The  first  two  variables  were  NAEP  region  (Northeast, 
Southeast,  Central,  and  West)  and  race/ethnicity  (White  nonHispanic,  Black  nonHispanic,  Hispanic,  and 
other).  For  fourth-grade  mathematics,  the  third  variable  was  mathematics  skill  (good,  not  sure,  other).  For 
eighth-grade  mathematics,  the  third  variable  was  the  student’s  mathematics  course  (eighth-grade 
mathematics,  pre-algebra,  algebra,  other).  For  eighth-grade  science,  the  third  variable  was  the  student’s 
science  course  (earth  science  only,  physical  science  only  or  only  earth  science  and  physical  science,  other). 
The  categorical  variables  and  control  totals  for  each  of  the  assessed  grades  and  subjects  are  presented  in 
Tables  10-12  and  10-13. 


Table  10-12 

First  and  Second  Categorical  Variables  Used  for  Raking^ 


Raking  Fourth  Grade  Mathematics 

Dimensions  Control  Total 

Eighth  Grade  Mathematics  Eighth  Grade  Science 
Control  Total  Control  Total 

First  Dimension 

NAEP  Region 

1 . Northeast 

631,451 

480,785 

475,079 

2.  Southeast 

733,191 

723,032 

726,322 

3.  Central 

427,577 

433,541 

432,580 

4.  West 

932,339 

904,369 

915,815 

Total 

2,724,558 

2,541,727 

2,549,797 

Second  Dimension 

Race/Ethnicity 

1.  White  nonHispanic 

1,690,310 

1,596,274 

1,590,164 

2.  Black  nonHispanic 

408,725 

377,660 

372,715 

3.  Hispanic 

454,883 

408,197 

419,467 

4.  Other 

170,641 

159,595 

167,451 

Total 

2,724,558 

2,541,727 

2,549,797 

* Numbers  may  not  add  up  exactly  due  to  rounding. 


The  equating  of  each  weight  distribution  was  achieved  using  a procedure  known  as  iterative 
proportional  fitting  (described  by  Little  & Rubin,  1987).  At  the  end  of  the  fitting,  adjustment  factors  were 
derived  and  multiplied  to  the  main  sample  weights  for  each  subgroup  to  force  their  distribution  to  agree 
with  that  from  the  aggregated  state  samples,  for  each  of  these  three  variables  in  turn.  This  process  was  then 
repeated,  and  the  final  set  of  adjusted  weights  was  compared  with  the  state  sample  weights  on  all  three 
distributions,  and  found  to  be  in  very  close  agreement.  Table  10-14  shows  the  distribution  of  the 
adjustment  factors  for  each  of  the  grades  and  subjects  assessed. 
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Table  10-13 

Third  Categorical  Variable  Used  for  Raking^ 


Control  Totals 

Grade  4 

Mathematics  Skill 

1.  Good 

1,734,093 

2.  Not  sure 

634,732 

3.  Other 

355,733 

Total 

2,724,558 

Grade  8 

Mathematics  Type  Course 

1 . Eight  Grade  Mathematics 

1,018,743 

2.  Pre-Algebra 

714,925 

3.  Algebra 

639,393 

4.  Other 

168,665 

Total 

2,541,727 

Science  Type  Course 

1 . Earth  Sciences  only 

737,898 

2.  Physical  Sciences  only  or  Earth  Science  and  Physical  Science 

525,048 

3.  Other 

1,286,851 

Total 

2,549,797 

' Numbers  may  not  add  up  exactly  due  to  rounding. 


Table  10-14 


Percentiles  of  Raking  Adjustments 


Distribution 

Grade  4 Mathematics 

Grade  8 Mathematics 

Grade  8 Science 

Minimum 

0.587 

0.708 

0.721  / 

10th  Percentile 

0.699 

0.741 

0.810 

25th  Percentile 

0.777 

0.837 

0.847 

Median 

0.891 

0.958 

0.983 

75th  Percentile 

1.025 

1.074 

1.161 

90th  Percentile 

1.119 

1.230 

1.409 

Maximum 

1.524 

1.728 

2.355 

10.2.5.2  School  Weights 

The  sampling  procedures  used  to  obtain  national  probability  samples  of  assessed  students  also 
gave  rise  indirectly  to  several  national  probability  samples  of  schools  (from  which  the  students  were 
subsequently  sampled).  So  that  the  school  samples  can  be  utilized  for  making  national  estimates  about 
schools,  appropriate  nonresponse  adjusted  survey  weights  have  been  developed. 

For  the  first  time  in  1996,  the  school  weights  for  the  main  assessments  were  computed  separately 
by  subject  within  grade.  The  school  weights  were  a direct  byproduct  of  the  student  weighting  process.  The 
weight  for  school  i in  session  h is  given  by 

Wm  = PSUWTi  • SCHWTi  • SESSWThi  • SESNRFm*  SCHPSFh, 
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where  PSUWTi,  SCHWTi,  SESSWT^i,  SESNRF^i,  and  SCHPSF,,,  are  defined  in  Section  10.2.1. 


The  school  weights  for  the  long-term  trend  assessments  in  1996  were  computed  using  the  same 
procedures  used  in  earlier  years.  The  weights  were  computed  separately  by  age  class.  The  school  base 
weight  was  the  product  of  PSUWT  and  SCHWT.  School  nonresponse  adjustments  were  then  applied  to 
these  base  weights.  The  values  of  the  adjustment  factors  are  not  subject  specific.  In  fact,  they  are  identical 
to  the  school  nonresponse  adjustment  factors  used  for  excluded  students.  Schools  that  did  not  participate  in 
any  of  the  sessions  that  they  were  assigned  were  treated  as  nonrespondents,  but  schools  that  conducted  at 

least  one  of  their  assigned  sessions  were  treated  as  respondents. 

\ 

Fifteen  samples  of  schools  were  weighted  to  be  nationally  representative  in  the  main  assessments, 
and  three  samples  were  weighted  to  be  nationally  representative  in  the  long-term  trend  assessments.  In  the 
main  samples,  the  population  of  schools  represented  is  that  of  schools  with  grade  4 (for  the  grade  4 
assessments),  grade  8 (for  the  grade  8 assessments),  or  grade  12  (for  the  grade  12  assessments).  In  the 
long-term  trend  samples,  the  school  population  at  age  class  9 is  that  of  schools  having  eligible  students  and 
at  least  one  of  the  grades  2 through  5,  the  school  population  at  age  class  13  is  that  of  schools  having 
eligible  students  and  at  least  one  of  the  grades  6 through  9,  while  the  school  population  at  age  class  17  is 
that  of  schools  having  eligible  students  and  at  least  one  of  the  grades  9 through  12. 


10.2.5.3  Jackknife  Replicate  Weights 

In  addition  to  the  weights  that  were  used  to  derive  all  estimates  of  population  and  subpopulation 
characteristics,  other  sets  of  weights,  called  jackknife  replicate  weights,  were  derived  to  facilitate  the 
estimation  of  sampling  variability  by  the  jackknife  variance  estimation  technique.  These  weights  and  the 
jackknife  estimator  are  discussed  in  the  next  section. 


10.3  PROCEDURES  USED  BY  NAEP  TO  ESTIMATE  SAMPLING  VARIABILITY 

A major  source  of  uncertainty  in  the  estimation  of  the  value  in  the  population  of  a variable  of 
interest  exists  because  information  about  the  variable  is  obtained  on  only  a sample  from  the  population.  To 
reflect  this  fact,  it  is  important  to  attach  to  any  statistic  (e.g.,  a mean)  an  estimate  of  the  sampling 
variability  to  be  expected  for  that  statistic.  Estimates  of  sampling  variability  provide  information  about 
how  much  the  value  of  a given  statistic  would  be  likely  to  change  if  the  statistic  had  been  based  on  another, 
equivalent,  sample  of  individuals  drawn  in  exactly  the  same  manner  as  the  achieved  sample. 

Another  important  source  of  variability  is  that  due  to  imprecision  in  the  measurement  of  individual 
proficiencies.  For  the  1996  assessment,  proficiencies  in  all  subject  areas  were  summarized  through  item 
response  theory  (IRT)  models,  but  not  in  the  way  that  these  models  are  used  in  standard  applications  where 
each  person  responds  to  enough  items  to  allow  for  precise  estimation  of  that  person’s  proficiency.  In 
NAEP,  each  individual  responds  to  relatively  few  items  so  that  individual  proficiency  values  are  not  well 
determined.  Consequently,  the  variance  of  any  statistic  based  on  proficiency  values  has  a component  due 
to  the  imprecision  in  the  measurement  of  the  proficiencies  of  the  sampled  individuals  in  addition  to  a 
component  measuring  sampling  variability.  The  estimation  of  the  component  of  variability,  due  to 
measurement  imprecision  and  its  effect  on  the  total  variability  of  statistics  based  on  proficiency  values  are 
discussed  in  Chapter  11. 
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The  estimation  of  the  sampling  variability  of  any  statistic  must  take  into  account  the  sample 
design.  In  particular,  because  of  the  effects  of  cluster  selection  (students  within  schools,  schools  within 
PSUs)  and  because  of  effects  of  nonresponse  and  poststratification  adjustments,  observations  made  on 
different  students  cannot  be  assumed  to  be  independent  of  each  other  (and  are,  in  fact,  generally  positively 
correlated).  Furthermore,  to  account  for  the  differential  probabilities  of  selection  (and  the  various 
adjustments),  each  student  has  an  associated  sampling  weight,  which  should  be  used  in  the  computation  of 
any  statistic  and  which  is  itself  subject  to  sampling  variability.  Ignoring  the  special  characteristics  of  the 
sample  design  and  treating  the  data  as  if  the  observations  were  independent  and  identically  distributed,  will 
generally  produce  underestimates  of  the  true  sampling  variability,  due  to  the  clustering  and  unequal 
sampling  weights. 

The  proper  estimation  of  the  sampling  variability  of  a statistic  based  on  the  NAEP  data  is 
complicated  and  requires  techniques  beyond  those  commonly  available  in  standard  statistical  packages. 
Fortunately,  the  jackknife  procedure  (see,  e.g.,  Wolter,  1985;  Kish  & Frankel,  1974;  Rust,  1985)  provides 
good  quality  estimates  of  the  sampling  variability  of  most  statistics,  at  the  expense  of  increased 
computation,  and  can  be  used  in  concert  with  standard  statistical  packages  to  obtain  a proper  estimate  of 
sampling  variability. 

The  jackknife  procedure  used  by  NAEP  has  a number  of  properties  that  make  it  particularly  suited 
for  the  analysis  of  NAEP  data.  When  properly  applied,  a jackknife  estimate  of  the  variability  of  a linear 
estimator  (such  as  a total)  will  be  the  same  as  the  standard  textbook  variance  estimate  specified  for  the 
sample  design  (if  the  first-stage  units  were  sampled  with  replacement  and  approximately  so  otherwise). 
Additionally,  if  the  finite  sampling  corrections  for  the  first  stage  units  can  be  ignored,  the  jackknife 
produces  asymptotically  consistent  variance  estimates  for  statistics  such  as  ratios,  regression  estimates  or 
weighted  means  and  for  any  other  nonlinear  statistic  that  can  be  expressed  as  a smooth  function  of 
estimated  totals  of  one  or  more  variables  (Krewski  & Rao,  1981). 

Through  the  creation  of  student  replicate  weights  (defined  below),  the  jackknife  procedure  allows 
the  measurement  of  variability  attributable  to  the  use  of  poststratification  and  other  weight  adjustment 
factors  that  are  dependent  upon  the  observed  sample  data.  Once  these  replicate  weights  are  derived,  it  is  a 
straightforward  matter  to  obtain  the  jackknife  variance  estimate  of  any  statistic. 

The  jackknife  procedure  in  this  application  is  based  upon  the  development  of  a set  of  jackknife 
replicate  weights  for  each  assessed  student  (or  excluded  student,  or  school  depending  upon  the  file 
involved).  The  replicate  weights  are  developed  in  such  a way  that,  when  utilized  as  described  below, 
approximately  unbiased  estimates  of  the  sampling  variance  of  an  estimate  result,  with  an  adequate  number 
of  degrees  of  freedom  to  be  useful  for  purposes  of  making  inferences  about  the  parameter  of  interest. 

The  estimated  sampling  variance  of  a parameter  estirnator  t is  the  sum  of  M squared  differences 
(where  M is  the  number  of  replicate  weights  developed): 

yar(0  = £(^'-0  , 

1 = 1 

where  t,  denotes  the  estimator  of  the  parameter  of  interest,  obtained  using  the  ith  set  of  replicate  weights, 
SRWTi,  in  place  of  the  original  sample  of  full  sample  estimates  WT. 

For  each  of  the  three  sample  types  (see  Section  3.4  and  Chapter  5 for  a description  of  the  three 
sample  types)  in  the  main  assessment  samples,  62  replicate  weights  were  developed  using  the  procedures 
outlined  below.  Similar  procedures  were  followed  for  the  long  term  trend  samples.  However,  since  those 
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samples  were  based  on  fewer  PSUs  (52  rather  than  94),  the  long-term  trend  samples  have  fewer  replicate 
weights  (36  instead  of  62).  Full  details  of  the  generation  of  replicate  weights  for  all  samples  are  given  in 
Wallace  and  Rust  (1999). 


Of  the  62  replicate  weights  formed  for  each  record  from  a main  assessment  sample,  36  act  to 
reflect  the  amount  of  sampling  variance  contributed  by  the  noncertainty  strata  of  PSUs,  with  the  remaining 
26  replicate  weights  reflecting  the  variance  contribution  of  the  certainty  PSU  samples. 


The  derivation  of  the  36  replicate  weights  reflecting  the  variance  of  the  noncertainty  PSUs 
involves  first  defining  pairs  of  PSUs  in  a manner  that  models  the  design  as  one  in  which  two  PSUs  are 
drawn  with  replacement  per  stratum.  This  definition  of  pairs  is  undertaken  in  a manner  closely  reflective  of 
the  actual  design,  in  that  PSUs  are  pairs  that  are  drawn  from  strata  within  the  same  subuniverse,  and  with 
similar  stratum  characteristics.  The  same  definition  of  pairs  was  used  for  each  of  the  age/grade  classes  in 
the  main  assessment,  since  all  were  drawn  from  the  same  sample  of  noncertainty  PSUs.  The  72 
noncertainty  PSUs,  drawn  one  from  each  of  72  strata,  were  formed  into  36  pairs  of  PSUs,  where  the  pairs 
were  composed  of  PSUs  from  adjacent  strata  within  each  subuniverse  (thus  the  strata  were  relatively 
similar  on  socioeconomic  characteristics  such  as  proportion  minority  population,  population  change  since 
1980,  per  capita  income,  educational  attainment,  and  unemployment  rate).  Whereas  the  actual  sample 
design  was  to  select  one  PSU  with  probability  proportional  to  size  from  each  of  72  strata,  for  variance 
estimation  purposes  the  design  is  regarded  as  calling  for  the  selection  of  two  PSUs  with  probability 
proportional  to  size  with  replacement  from  each  of  36  strata.  This  procedure  likely  gives  a small  positive 
bias  to  estimates  of  sampling  error. 

The  student  replicate  weight  for  the  pair  of  noncertainty  PSUs,  for  the  36  pairs  corresponding  to 
values  of  i from  1 to  36,  is  computed  as  follows: 

1.  Let  Wb  be  the  base  weight  of  a student,  as  described  in  Section  10.2.1,  which  accounts 
for  the  various  components  of  the  selection  probability  for  the  student. 


2.  At  random,  one  PSU  in  each  pair  is  denoted  as  PSU  number  1,  while  the  other  is 
denoted  as  PSU  number  2.  The  replicate  base  weight  VV),,  is  given  by: 


0 

Wm  = {2*  Wb 
WB 


if  the  student  belongs  to  PSU  number  1 of  pair  i 
if  the  student  belongs  to  PSU  number  2 of  pair  i 
if  the  student  is  from  neither  PSU  in  pair  i 


3.  The  j*  student  replicate  weight  SEWT,  is  obtained  by  applying  the  various  school  and 
student  nonresponse  adjustments,  the  weight  trimming,  and  the  poststratification  to  the 
j*  set  of  replicate  base  weights,  using  procedures  identical  to  those  used  to  obtain  the 
final  student  weights  WT  from  the  set  of  base  weights  Wb- 


In  brief,  the  procedure  for  deriving  the  sets  of  W),,  values  from  the  Wb  values  reflects  the  sampling 
of  PSUs,  schools,  sessions,  and  students.  By  repeating  the  various  weight  adjustment  procedures  in  each 
set  of  replicate  base  weights,  the  impact  of  these  procedures  on  the  sampling  variance  of  the  estimator,  t,  is 
appropriately  reflected  in  the  variance  estimator  Var{i)  defined  above. 

The  procedure  for  obtaining  the  26  sets  of  replicate  weights  to  estimate  the  sampling  variance  from 
the  certainty  PSUs  is  analogous,  but  somewhat  more  complex.  The  first  stage  of  sampling  in  this  case  is  at 
the  school  level,  and  the  derivation  of  replicate  weights  must  reflect  appropriately  the  sampling  of  schools 
within  certainty  PSUs.  Since  each  of  the  three  grade  classes  in  the  main  assessment  involved  different 
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samples  of  schools,  the  procedure  for  forming  replicate  base  weights  was  individualized  to  each  of  these 
sample  components.  In  common  across  these  three  samples  were  the  22  certainty  PSUs  used,  and  the  fact 
that  26  replicate  weights  were  formed  in  each  case. 

For  a given  sample,  the  22  certainty  PSUs  constituted  strata,  with  a sample  of  schools  drawn 
systematically  within  each.  Using  the  schools  listed  in  order  of  sample  selection  within  each  stratum, 
successive  schools  were  paired  or  formed  into  triples.  These  pairs  and  triples  numbered  more  than  26,  so 
that  each  replicate  weight  was  in  general  formed  by  perturbing  the  weights  of  students  from  more  than  a 
single  pair  or  triple.  These  aggregates  of  pairs  and  triples  were  in  general  assigned  in  proportion  to  the  size 
of  the  PSU.  Thus  generally  speaking,  the  largest  PSUs  were  assigned  three  replicates  each,  the  next  largest 
were  assigned  two  replicates  each,  and  the  remaining  self-representing  PSUs  were  assigned  one  replicate 
each.  When  splitting  the  larger  PSUs,  the  schools  were  split  into  groups  of  (as  close  as  possible)  equal  size, 
based  on  the  ordering  at  the  time  of  sample  selection.  One  group  was  assigned  to  each  replicate.  Within 
each  PSU  (or  partial  PSU  in  the  case  of  the  large  split  PSUs)  schools  were  alternately  numbered  1 or  2 
starting  randomly.  If,  however,  there  were  exactly  three  schools  sampled  in  the  PSU  the  schools  were 
randomly  numbered  1,  2,  or  3.  The  method  of  forming  replicate  base  weights  in  strata  where  there  were  not 
exactly  three  schools  was  the  same  as  for  the  noncertainty  strata  (except  that  members  of  a pair,  i,  could 
come  from  more  than  a single  “stratum”). 

When  a stratum  contained  three  schools,  students  in  these  schools  had  their  weights  perturbed  for 
two  sets  of  replicates,  say  iT  and  i2,  as  follows: 


The  actual  pattern  of  replicate  base  weight  assignrrient  used  for  each  of  the  samples  is  given  in 
Wallace  and  Rust  (1999). 

The  nonresponse,  trimming,  and  poststratification  adjustments  were  applied  to  each  set  of  replicate 
base  weights  to  derive  the  final  replicate  weights  in  each  case,  exactly  as  in  the  noncertainty  PSUs.  In  fact 
these  procedures  were  applied  to  the  full  set  of  weights  from  all  parts  of  the  given  sample  together,  just  as 
for  the  full  sample  weights.  That  is,  for  example,  poststratification  factors  were  derived  from  the  full  set  of 
data  for  each  replicate,  not  separately  for  certainty  and  noncertainty  PSUs. 

This  estimation  technique  was  used  by  NAEP  to  estimate  all  sampling  errors  presented  in  the 
various  reports.  A further  discussion  of  the  variance  estimation  procedure  used  by  NAEP,  including  a 
discussion  of  alternative  jackknife  estimators  that  were  also  considered,  appears  in  Johnson  (1989). 

We  noted  above  (as  discussed  in  Chapter  1 1)  that  a separate  estimate  of  the  contribution  to 
variance  due  to  the  imprecision  in  the  measure  of  individual  proficiencies  is  made  and  added  to  the 
jackknife  estimate  of  variance.  That  variance  component  could  have  been  approximately  reflected  in  the 
jackknife  variance  estimates  simply  by  separately  applying  the  IRT  computations  to  each  jackknife 
replicate.  Because  of  the  heavier  IRT  computational  load,  this  was  not  done.  Less  work  was  involved  by 
the  simple  procedure  of  making  separate  estimates  of  this  component  to  be  added  to  the  jackknife  variance 


0 if  the  student  is  in  school  number  1 of  a PSU  belonging  to  set  i 


Wm  = < 1.5  * Wh  if  the  student  is  in  school  number  2 or  3 of  a PSU  belonging  to  set  i 

Wfl  if  the  student  does  not  belong  to  a PSU  in  set  i 


Ww  = < 0 


1.5  * Wb  if  the  student  is  in  school  number  1 or  2 of  a PSU  belonging  to  set  i 

0 if  the  student  is  in  school  number  3 of  a PSU  belonging  to  set  i 

VPfl  if  the  student  does  not  belong  to  a PSU  in  set  i 
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estimates.  Also,  a separate  measure  of  this  component  of  variance  is  then  available,  which  would  not  be  so 
if  it  were  reflected  in  the  jackknife  variance  estimate. 


10.4  APPROXIMATING  THE  SAMPLING  VARIANCE  USING  DESIGN  EFFECTS 

In  practical  terms,  the  major  expenditure  of  resources  in  the  computation  of  a jackknife  variance 
estimate  occurs  in  the  preparation  of  estimates  for  each  of  the  pseudoreplicates.  In  the  1996  assessment, 
this  implies  that  the  statistic  of  interest  has  to  be  recomputed  up  to  63  times,  once  for  the  overall  estimate  t, 
and  once  for  each  of  the  up  to  62  pseudoreplicates  ti.  Because  this  is  a considerable  increase  in  the  amount 
of  computation  required,  relative  to  a conventional  variance  estimate,  it  is  of  interest  to  see  how  much  the 
jackknife  variance  estimates  differ  from  their  less  computationally  intensive,  simple  random  sampling 
based,  analogues. 

The  comparison  of  the  conventional  and  the  jackknife  methods  of  variance  estimation  will  be  in 
terms  of  a statistic  called  the  design  effect,  which  was  developed  by  Kish  (1965)  and  extended  by  Kish  and 
Frankel  (1974).  The  design  effect  for  a statistic  is  the  ratio  of  the  actual  variance  of  the  statistic  (taking  the 
sample  design  into  account)  over  the  conventional  variance  estimate  based  on  a simple  random  sample 
with  the  same  number  of  elements.  The  design  effect  is  the  inflation  factor  to  be  applied  to  the 
conventional  variance  estimate  in  order  to  adjust  error  estimates  based  on  simple  random  sampling 
assumptions  to  account  approximately  for  the  effect  of  the  sample  design.  The  value  of  the  design  effect 
depends  on  the  type  of  statistic  computed  and  the  variables  considered  in  a particular  analysis  as  well  as 
the  combined  clustering,  stratification,  and  weighting  effects  occurring  among  sampled  elements.  While 
stratification  drives  down  the  sampling  variance,  the  effects  of  clustering  and  weighting  that  drive 
variances  up  are  generally  sufficient  to  produce  variance  estimates  that  are  larger  than  variances  based  on 
simple  random  sampling  assumptions.  Consequently,  the  design  effects  will  be  greater  than  one.  In  NAEP, 
the  underestimates  are  the  result  of  ignoring  the  effects  of  clustering  and  unequal  probabilities  of  selection 
in  the  variance  calculations. 

Since  most  of  the  analyses  conducted  by  NAEP  are  based  on  the  results  of  scaling  models  that 
summarize  performance  of  students  across  a learning  area,  we  consider  the  design  effects  to  be  expected 
for  analyses  based  on  these  scale  scores.  For  reasons  given  in  Chapter  11,  NAEP  provides  each  individual 
with  a set  of  “plausible  values,”  each  of  which  is  a random  draw  from  the  distribution  of  the  potential  scale 
scores  for  that  individual.  Since  our  current  interest  is  on  the  effect  of  the  sampling  design  on  estimation 
and  inference,  we  will  restrict  our  attention  to  a single  measure  of  an  individual’s  proficiency,  the  first 
plausible  value  of  the  individual’s  scale  score. 

A key  statistic  of  interest  is  the  estimated  mean  proficiency  of  a subgroup  of  the  population.  An 
estimate  of  the  subgroup  mean  proficiency  is  the  weighted  mean  of  the  first  plausible  values  of  proficiency 
of  the  sampled  individuals  who  belong  to  the  subpopulation  of  interest.  Let  Y be  the  weighted  mean  of  the 
plausible  values  of  the  sampled  members  of  the  subpopulation.  The  conventional  estimate  of  the  variance 
of  Y is 


— \2 


Varc.,n{Y)  = -^ 


Xw,(y,-y) 


N 


where  N is  the  total  number  of  sampled  individuals  in  the  subpopulation  for  which  plausible  values  are 
available,  w,  is  the  weight  of  the  individual,  y,  is  a plausible  value  from  the  distribution  of  potential 
proficiencies  for  that  individual,  and  W+  is  the  sum  of  the  weights  across  the  N individuals. 
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The  design  effect  for  the  subgroup  mean  proficiency  estimate  is 

dejf{Y)  = VarjK(Y)/Varcon(Y) 

where  VARjk(Y)  is  the  jackknife  variance  of  Y_^  (As  has  been  pointed  out  previously,  VARjk{Y)  as 
computed  does  not  measure  the  variability  of  Y due  to  imprecision  in  the  measurement  of  the  proficiencies 
of  the  sampled  individuals.  The  estimation  of  this  very  important  source  of  variability  is  discussed  in 
Chapter  1 1 .)  Of  the  factors  that  determine  dejf(Y),  the  effects  of  stratification  are  usually  less  than  one, 
which  means  the  efficiency  of  a stratified  sampling  is  better  than  a simple  random  sampling;  whereas  the 
clustering  effects  are  always  larger  than  one.  The  clustering  effects  can  be  approximated  by 

l + (m-l)p, 

where  m is  the  average  cluster  size  and  p is  the  intracluster  correlation  (Cochran,  1977).  Therefore,  the 
large  cluster  size  or  large  intracluster  correlation  will  inflate  the  clustering  effects. 

Values  of  the  design  effects  for  subgroup  mean  proficiencies  are  displayed,  by  grade,  in  Tables  10- 
15  and  10-19  for  the  1996  main  assessments  of  mathematics  and  science  respectively.  Design  effects  are 
shown  for  the  population  as  a whole  (Total)  as  well  as  for  a variety  of  demographic  subgroups;  gender; 
race/ethnicity  (White,  Black,  Hispanic,  Asian/Pacific  Islander,  other);  type  of  location  (Central  City,  Urban 
Fringe/Large  Town,  Rural/Small  Town);  parental  education  (did  not  graduate  high  school,  graduated  high 
school,  post-high  school,  graduated  college,  unknown);  and  type  of  school  (public,  nonpublic).  These 
particular  demographic  variables  were  selected  because  (1)  they  are  major  variables  in  NAEP  reports  and 
(2)  they  reflect  different  types  of  divisions  of  the  population  that  might  have  different  levels  of  sampling 
variability.  Note  that  the  tables  of  the  design  effects  provided  in  the  NAEP  Technical  Reports  previous  to 
1994  are  computed  for  the  mean  item  scores,  proportion-correct  statistics,  which  can  not  be  compared  with 
the  design  effects  for  proficiency  scale  scores  directly. 

The  1996  main  mathematics  assessment  contains  the  three  sample  types  SI,  S2  and  S3  (see 
Section  5.3).  To  conserve  trend  in  the  main  mathematics  assessment,  the  reporting  samples  were  made  up 
of  A1  and  B1  portions  in  SI  samples  and  A2  in  S2  samples  (see  Table  10-1).  The  advantage  of  including 
A2  in  the  reporting  samples  is  to  obtain  more  accurate  scale  scores,  yet  a trade-off  is  that  the  clustering 
effects  for  the  reporting  samples  becomes  lager  than  the  clustering  effects  for  only  sample  S 1 . 

The  larger  intracluster  correlation  in  A1  and  A2  and  larger  cluster  sizes  in  sample  type  SI  are  two 
factors  that  contribute  to  the  increase  in  clustering  effects.  First,  the  reporting  sample  for  the  main 
mathematics  assessment  from  sample  type  S2  has  only  non-SD/LEP  students,  A2.  Compared  with  the 
SD/LEP  students,  the  non-SD/LEP  students  are  relative  homogenous  in  scale  scores.  The  high 
homogeneity  in  clusters  implies  a large  intracluster  correlation,  p 1 . Given  other  conditions,  the  large 
clustering  effects  that  are  due  to  a large  intracluster  correlation  expand  the  design  effects  in  the  reporting 
samples.  To  check  this  conclusion,  the  design  effects  for  subsamples  of  Al,  A2,  and  B1  were  calculated 
separately  and  are  displayed  in  Tables  10-16,  10-17  and  10-18.  The  estimated  design  effects  for 
subsamples  of  Al  and  A2  are  much  larger  than  those  for  subsample  B1  that  contains  SD/LEP  students  in 
sample  type  S 1 . Secondly,  the  cluster  sizes  tended  to  be  larger  in  sample  type  SI  because  the  schools  in 
sample  type  SI  only  provided  samples  for  the  main  mathematics  assessment,  whereas  the  schools  in 
sample  type  S2  provided  samples  for  both  the  main  mathematics  and  science  assessments.  Therefore,  the 
design  effects  in  Al  are  found  to  be  larger  than  the  design  effects  in  A2  and  both  subsamples  of  Al  and  A2 
contributed  students  to  the  reporting  samples. 
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Tables  10-20  and  10-21  provide  equivalent  information  for  the  long-term  trend  samples.  Table. 10-20 
provides,  for  each  age  class  and  demographic  subgroup,  the  average  of  the  design  effects  for  mean  reading 
and  writing  proficiencies  for  the  students  selected  for  the  long-term  trend  assessments  of  reading  and 
writing.  Table  10-21  provides  the  average  of  the  design  effects  for  the  mean  mathematics  and  science 
proficiencies  for  the  students  selected  for  the  long-term  trend  assessments  of  mathematics  and  science. 

Finally,  for  comparison  with  the  national  mathematics  and  science  results.  Table  10-22  shows  the 
average  design  effects  for  state-level  mean  mathematics  proficiency,  averaged  across  all  jurisdictions 
participating  in  the  grade  4 and  grade  8 1996  State  Assessment  in  mathematics.  The  results  in  Table  10-23 
are  the  average  design  effects  for  state-level  mean  science  proficiency,  averaged  across  all  jurisdictions 
participating  in  the  grade  8 1996  State  Assessment  in  science. 

The  tables  show  that  the  design  effects  are  predominantly  larger  than  1,  indicating  that  standard  variance 
estimation  formulas  will  be  generally  too  small,  usually  markedly  so.  Although  the  design  effects  appear 
somewhat  different  for  certain  subgroups  of  the  population,  they  are,  perhaps,  similar  enough  (at  least  within 
a subject  and  grade)  to  select  an  overall  composite  value  that  is  adequate  for  most  purposes.  In  choosing  a. 
composite  design  effect,  some  consideration  must  be  made  about  the  relative  consequences  of 
overestimating  the  variance  as  opposed  to  underestimating  the  variance.  For  example,  if  an  overestimate  of 
the  variance  is  viewed  as  severe  an  error  as  an  underestimate,  the  composite  design  effect  should  be  near  to 
the  center  of  the  distributions  of  the  design  effects.  Possible  composites  of  this  type  are  the  mean  and 
median  design  effects  across  the  combined  distribution  of  all  design  effects.  Larger  design  effects  should  be 
used  if  it  is  felt  that  it  is  a graver  error  to  underestimate  the  variability  of  a statistic  than  to  overestimate  it. 
For  example,  Johnson  and  King  (1987)  examine  estimation  of  variances  using  design  effects  (among  other 
techniques)  under  the  assumption  that  the  consequences  of  an  underestimate  are  three  times  as  severe  as 
those  of  an  overestimate  of  the  same  magnitude.  Adopting  a loss  function  that  is  a weighted  sum  of  absolute 
values  of  the  deviations  of  predicted  from  actual  with  underestimates  receiving  three  times  the  weight  of 
overestimates,  produces  the  upper  quartile  of  the  design  effects  as  the  composite  value.  This  assumes  that 
the  distribution  of  design  effects  is  roughly  independent  of  the  jackknife  estimates  of  variance,  so  that  the 
size  of  a design  effect  does  not  depend  on  the  size  of  the  variance. 

Table  10-24  gives  the  values  of  these  potential  composites,  by  grade,  for  the  mathematics  and 
science  assessments,  and  across  those  assessments.  Tables  10-25  and  10-26  gives  composite  values  for  the 
1996  State  Assessment  of  mathematics  (grades  4 and  8),  and  the  1996  State  Assessment  of  science  (grade  8), 
respectively.  Table  10-27  shows  composite  values  for  the  1996  State  Assessment  of  mathematics  and  the 
1996  State  Assessment  of  science,  combined.  The  state  assessments  tend  to  have  smaller  design  effects  than 
the  matching  national  asse^ment,  due  to  the  lesser  degree  of  clustering  in  the  state  assessment  samples  (i.e., 
the  average  cluster  size  of  m is  smaller).  Table  10-28  gives  the  values  of  the  composites  for  the  two  long- 
term trend  samples. 

We  note  that  the  Varcon{Y^  as  defined  above  is  an  estimate  of  S^/N  where  represents  the  unit 

variance  for  a simple  random  sample  for  the  population  of  students  from  which  the  sample  is  drawn.  This  is 
an  appropriate  estimate  of  the  increase  in  variance  over  simple  random  sampling  from  that  population  due  to 
the  effects  of  weighting.  However,  the  computer  packages  used  for  estimating  the  variance  may  not  reflect 
the  weights  in  estimating  the  unit  variance,  as  given  above,  but  instead  may  provide  an  estimate  of  a unit 
variance  of  the  form 

In  this  case,  the  unweighted  estimate  of  unit  variance  would  be  appropriate  for  the  denominator  of  a design 
effect  measure  of  the  increase  in  variance  over  the  unit  variance  as  estimated  by  the  computer  package.  If 


there  is  no  correlation  between  the  w,  and  there  would  be  little  difference  between  the  two. 


Table  10-15 


Design  Effects  by  Demographic  Subgroup  and  Grade 
for  Mean  Mathematics  Proficiencies' 


Subgroup 

Grade  4 

Grade  8 

Grade  12 

Total 

5.00 

5.82 

6.50 

Male 

3.28 

4.48 

3.50 

Female 

3.19 

3.13 

4.56 

White 

4.74 

5.92 

5.14 

Black 

6.32 

3.89 

5.95 

Hispanic 

4.04 

2.72 

2.42 

Asian/Pacific  Islander 

3.86 

4.45 

6.21 

Other  race/ethnicity 

1.46 

1.09 

15.38 

Urban 

9.29 

7.27 

11.53 

Suburban 

7.10 

10.50 

6.36 

Rural 

6.09 

5.92 

8.00 

PARED  <HS 

1.54 

1.33 

1.59 

PARED  = HS 

2.12 

1.89 

2.28 

PARED  > HS 

1.18 

2.34 

1.31 

PARED  = College 

4.76 

5.05 

5.17 

PARED  = Unknown 

4.36 

1.13 

1.06 

Public  school 

5.24 

5.87 

4.42 

Nonpublic  school 

7.00 

8.53 

18.24 

'Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 


Table  10-16 

Design  Effects  of  Subsample  A1  by  Demographic  Subgroup  and  Grade 
for  Mean  Mathematics  Proficiencies' 


Subgroup 

Grade  4 

Grade  8 

Grade  12 

Total 

9.38 

7.33 

7.48 

Male 

5.61 

5.04 

4.62 

Female 

4.95 

4.13 

4.14 

White 

5.13 

6.89 

5.21 

Black 

15.14 

3.93 

7.63 

Hispanic 

2.08 

2.86 

1.91 

Asian/Pacific  Islander 

2.46 

7.96 

4.54 

Other  race/ethnicity 

1.12 

0.96 

30.13 

Urban 

16.94 

6.13 

12.20 

Suburban 

6.84 

15.34 

5.72 

Rural 

8.25 

8.01 

13.42 

PARED  < HS 

1.28 

1.55 

1.03 

PARED  = HS 

2.70 

1.76 

2.74 

PARED  > HS 

1.32 

2.28 

1.57 

PARED  = College 

7.92 

7.96 

5.88 

^PARED  = Unknown 

— 

— 

— 

Public  school 

9.49 

7.07 

5.43 

Nonpublic  school 

12.24 

9.18 

20.54 

' Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 
^ Insufficient  data  to  compute  design  effects 
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Table  10-17 

Design  Effects  of  Subsample  A2  by  Demographic  Subgroup  and  Grade 
for  Mean  Mathematics  Proficiencies^ 


Subgroup 

Grade  4 

Grade  8 

Grade  12 

Total 

7.95 

3.98 

5.98 

Male 

4.75 

3.50 

3.19 

Female 

4.78 

2.02 

5.32 

White 

4.39 

3.95 

3.50 

Black 

3.43 

1.80 

1.92 

Hispanic 

3.26 

2.31 

3.00 

Asian/Pacific  Islander 

3.92 

2.07 

1.61 

Other  race/ethnicity 

0.60 

1.41 

2.05 

Urban 

7.25 

6.77 

5.53 

Suburban 

13.77 

4.80 

5.36 

Rural 

4.24 

4.25 

3.94 

PARED  < HS 

1.44 

1.52 

1.76 

PARED  = HS 

1.81 

2.18 

1.41 

PARED  > HS 

1.19 

0.92 

1.89 

PARED  = College 

5.55 

2.10 

3.27 

^PARED  = Unknown 

— 

— 

— 

Public  school 

7.79 

4.02 

5.08 

Nonpublic  school 

5.99 

7.58 

6.08 

* Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 
^ Insufficient  data  to  compute  design  effects 


Table  10-18 

Design  Effects  of  Subsample  B1  by  Demographic  Subgroup  and  Grade 
for  Mean  Mathematics  Proficiencies^ 


Subgroup 

Grade  4 

Grade  8 

Grade  12 

Total 

2.88  • 

1.23 

1.77 

Male 

2.42 

1.38 

1.38 

Female 

1.31 

1.15 

1.98 

White 

1.95 

1.54 

1.10 

Black 

1.60 

0.98 

1.77 

Hispanic 

3.73 

0.81 

0.78 

Asian/Pacific  Islander 

1.90 

0.80 

8.32 

Other  race/ethnicity 

0.87 

1.13 

13.01 

Urban 

3.96 

1.26 

1.91 

Suburban 

2.76 

1.55 

1.19 

Rural 

1.67 

1.27 

1.86 

PARED  < HS 

1.38 

0.69 

1.46 

PARED  = HS 

1.46 

0.98 

0.89 

PARED  > HS 

1.21 

0.98 

1.21 

PARED  = College 

1.45 

2.00 

0.72 

^PARED  = Unknown 

— 

— 

— 

Public  school 

2.85 

1.10 

1.49 

Nonpublic  school 

0.31 

0.35 

5.99 

' Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 
^ Insufficient  data  to  compute  design  effects 
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Table  10-19 

Design  Effects  by  Demographic  Subgroup  and  Grade 
for  Mean  Science  Proficiencies^ 


Subgroup 

Grade  4 

Grade  8 

Grade  12 

Total 

3.83 

4.98 

4.80 

Male 

2.43 

3.35 

3.77 

Female 

2.43 

4.23 

3.25 

White 

3.71 

5.09 

4.73 

Black 

4.17 

2.01 

2.79 

Hispanic 

3.08 

3.57 

4.60 

Asian/Pacific  Islander 

4.00 

3.07 

2.81 

Other  race/ethnicity 

2.45 

2.38 

2.18 

Urban 

11.40 

7.17 

11.86 

Suburban 

13.97 

8.51 

8.60 

Rural 

6.17 

6.66 

5.07 

PARED  < HS 

1.23 

1.75 

1,91 

PARED  = HS 

1.79 

2.35 

3.11 

PARED  > HS 

1.16 

1.66 

2.43 

PARED  = College 

2.64 

4.50 

2.95 

PARED  = Unknown 

2.77 

3.72 

1.74 

Public  school 

3.97 

4.43 

4.94 

Nonpublic  school 

5.23 

9.31 

6.96 

^ Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 


Table  10-20 

Design  Effects  by  Demographic  Subgroup  and  Age  Averaged  Over  Mean 
Reading  and  Writing  Proficiencies  for  the  Reading  and  Writing  Long-Term  Trend  Samples^ 


Subgroup 

Age  9 

Age  13 

Age  17 

Total 

1.86 

2.13 

2.13 

Male 

1.08 

1.81 

1.79 

Female 

2.00 

1.85 

2.08 

White 

1.51 

1.24 

1.84 

Black 

1.31 

3.93 

2.48 

Hispanic 

2.17 

1.98 

1.21 

Asian/Pacific  Islander 

1.53 

1.78 

2.22 

Other  race/ethnicity 

1.23 

1.75 

3.75 

Urban 

2.16 

6.65 

2.58 

Suburban 

3.37 

4.35 

2.08 

Rural 

3.74 

3.33 

8.14 

PARED  < HS 

0.89 

1.66 

1.06 

PARED  = HS 

1.68 

1.09 

1.03 

PARED  >HS 

1.12 

1.22 

1.86 

PARED  = College 

1.21 

2.09 

1.42  . 

PARED  = Unknown 

2.08 

2.04 

0.68 

Public  school 

2.05 

1.89 

2.41 

Nonpublic  school 

1.96 

2.28 

2.16 

' Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 
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Table  10-21 

Design  Effects  by  Demographic  Subgroup  and  Age  Averaged  Over  Mean 
Mathematics  and  Science  Proficiencies  for  the  Mathematics  and  Science  Long-Term  Trend  Samples^ 


Subgroup 

Age  9 

Age  13 

Age  17 

Total 

3.02 

3.54 

2.04 

Male 

2.88 

1.76 

1.86 

Female 

1.37 

2.83 

1.45 

White 

2.49 

3.95 

2.04 

Black 

2.83 

1.79 

1.17 

Hispanic 

2.06 

2.06 

1.75 

Asian/Pacific  Islander 

2.29 

2.72 

1.71 

Other  race/ethnicity 

1.79 

2.18 

1.8 

Urban 

4.42 

6.18 

5.16 

Suburban 

4.36 

7.73 

2.42 

Rural 

6.32 

8.52 

6.35 

PARED  < HS 

0.84 

2.09 

1.8 

PARED  = HS 

1.51 

1.79 

2.39 

PARED  > HS 

1.88 

1.08 

1.7 

PARED  = College 

2.67 

2.14 

1.77 

PARED  = Unknown 

0.71 

1.86 

1.47 

Public  school 

2.72 

3.57 

1.86 

Nonpublic  school 

4.36 

10.89 

4.74 

* Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of  proficiency. 


Table  10-22 

Average  Design  Effects  by  Demographic  Subgroup  for  Mean 
State  Mathematics  Proficiencies  Averaged  Across  State  Samples^ 


Subgroup 

Grade  4 

Grade  8 

Total 

4.15 

3.45 

Male 

2.65 

2.36 

Female 

2.66 

2.36 

White 

3.06 

2.56 

Black 

1.98 

1.93 

Hispanic 

1.84 

1.59 

Asian/Pacific  Islander 

1.57 

1.40 

Other  race/ethnicity 

1.63 

1.76 

Urban 

5.99 

4.88 

Suburban 

4.23 

3.59 

Rural 

4.19 

3.08 

PARED  < HS 

1.37 

1.26 

PARED  = HS 

1.65 

1.79 

PARED  > HS 

1.37 

1.44 

PARED  = College 

2.89 

2.37 

PARED  = Unknown 

2.13 

1.47 

Public  school 

4.02 

3.42 

Nonpublic  school 

7.06 

5.60 

’ Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means 
of  the  first  plausible  values  of  proficiency. 
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Table  10-23 

Average  Design  Effects  by  Demographic  Subgroup  for  Mean 
State  Science  Proficiencies  Averaged  Across  State  Samples^ 


Subgroup 

Grade  8 

Total 

3.53 

Male 

2.43 

Female 

2.33 

White 

2.55 

Black 

1.92 

Hispanic 

1.63 

Asian/Pacific  Islander 

1.34 

Other  race/ethnicity 

1.70 

Urban 

5.13 

Suburban 

3.52 

Rural 

2.99 

PARED  < HS 

1.34 

PARED  = HS 

1.75 

PARED  > HS 

1.45 

PARED  = College 

2.26 

PARED  = Unknown 

1.51 

Public  school 

3.41 

Nonpublic  school 

6.50 

^ Design  effects  are  based  on  the  conventional  and  jackknife  variances  of 
subgroup  means  of  the  first  plausible  values  of  proficiency. 


Table  10-24 

Within-Grade  Mean,  Median,  and  Upper  Quartile  of  the  Distribution  of  Design  Effects 
for  National  Main  Assessments  by  Subject  Area  and  Across  Subject  Areas 

Statistic  Grade  4 Grade  8 Grade  12 


Mean  Mathematics  Proficiencies 
(Distribution  Across  Demographics  Subgroups) 


Upper  Quartile 

6.09 

5.92 

6.50 

Mean 

4.48 

4.52 

6.09 

Median 

4.88 

5.44 

6.08 

Mean  Science  Proficiencies 

(Distribution  Across  Demographics  Subgroups) 

Upper  Quartile 

4.17 

5.09 

4.94 

Mean 

4.25 

4.37 

4.36 

Median 

3.90 

4.47 

4.67 

Across  Subject  Areas 

(Distribution  Across  Subject  Areas  and  Demographic  Subgroups) 

Upper  Quartile 

5.24 

5.92 

6.36 

Mean 

4.36 

4.45 

5.23 

Median 

3.92  ■ 

4.33 

4.58 

ERIC 
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Table  10-25 

Mean,  Median,  and  Upper  Quartile  of  the  Across-State  Average  Design  Effects 
for  Mean  State  Mathematics  Proficiency  (Distribution  Across  Demographic  Subgroups) 


Statistics 

Grade  4 

Grade  8 

Upper  Quartile 

4.15 

3.42 

Mean 

3.02 

2.57 

Median 

2.66 

2.36 

Table  10-26 

Mean,  Median,  and  Upper  Quartile  of  the  Across-State  Average  Design  Effects 
for  Mean  State  Science  Proficiency  (Distribution  Across  Demographic  Subgroups) 

Statistics 

Grade  8 

Upper  Quartile 

3.41 

Mean 

2.63 

Median 

2.30 

Table  10-27 

Mean,  Median,  and  Upper  Quartile  of  the  Across-State  Average  Design  Effects 
for  Mean  State  Proficiency  (Distribution  Across  Demographic  Subgroups  and  Across  Subjects) 


Statistics 

Grade  4 

Grade  8 

Upper  Quartile 

4.15 

3.42 

Mean 

3.02 

2.60 

Median 

2.66 

2.35 

Table  10-28 

Mean,  Median,  and  Upper  Quartile  of  the  Distribution  of  Design  Effects 
for  the  Long-Term  Trend  Samples' 


Statistic 

Age  9 

Age  13 

Age  17 

Reading  and  Writing  Long-Term  Trend 
(Distribution  Across  Demographic  Subgroups  of  Average 
of  Design  Effects  for  Reading  and  Writing  Mean  Proficiencies) 
Upper  Quartile 

2.08 

3.09 

2.48 

Mean 

1.81 

2.44 

2.40 

Median 

1.67 

1.94 

2.08 

Mathematics  and  Science  Long-Term  Trend 
(Distribution  Across  Demographic  Subgroups  of  Average 
of  Design  Effects  for  Mathematics  and  Science  Mean  Proficiencies) 
Upper  Quartile 

3.02 

3.95 

2.39 

Mean 

2.70 

3.70 

2.42 

Median 

2.58 

2.45 

1.83 

' Design  effects  are  based  on  the  conventional  and  jackknife  variances  of  subgroup  means  of  the  first  plausible  values  of 
proficiency. 
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Chapter  11 

SCALING  PROCEDURES^ 

Nancy  L.  Allen,  Eugene  G.  Johnson,  Robert  J.  Mislevy,  and  Neal  Thomas 

Educational  Testing  Service 


11.1  INTRODUCTION 

The  primary  method  by  which  results  from  the  1996  National  Assessment  of  Educational 
Progress  (NAEP)  were  disseminated  was  scale-score  reporting.  With  scaling  methods,  the  performance 
of  a sample  of  students  in  a subject  area  or  subarea' can  be  summarized  on  a single  scale  or  series  of 
scales  even  when  different  students  have  been  administered  different  items.  This  chapter  presents  an 
overview  of  the  scaling  methodologies  employed  in  the  analyses  of  the  data  from  NAEP  surveys  in 
general  and  from  the  1996  assessment  in  particular.  Details  of  the  scaling  procedures  specific  to  the 
subject  areas  of  science,  mathematics,  reading,  and  writing  are  presented  in  Chapters  12  through  17. 


11.2  BACKGROUND 

The  basic  information  from  an  assessment  consists  of  the  responses  of  students  to  the  items 
presented  in  the  assessment.  For  NAEP,  these  items  are  constructed  to  measure  perforniance  on  sets  of 
objectives  developed  by  nationally  representative  panels  of  learning  area  specialists,  educators,  and 
concerned  citizens.  Satisfying  the  objectives  of  the  assessment  and  ensuring  that  the  tasks  selected  to 
measure  each  goal  cover  a range  of  difficulty  levels  typically  require  many  items.  For  example,  the 
mathematics  assessment  required  164  items  at  grade  8.  Depending  on  the  subject  areas,  a mixture  of 
multiple-choice,  short  constructed-response,  and  extended  constructed-response  items  were  used. 
Multiple-choice  and  short  constructed-response  items  were  used  in  all  assessments  but  writing.  Extended 
constructed-response  items,  scored  on  a multipoint  scale,  were  presented  in  the  main  mathematics  and 
science  assessments  and  in  the  long-term  trend  writing  assessment.  To  reduce  student  burden,  each 
assessed  student  was  presented  only  a fraction  of  the  full  pool  of  items  through  multiple  matrix  sampling 
procedures. 

The  most  direct  manner  of  presenting  the  assessment  results  is  to  report  separate  statistics  for 
each  item.  However,  because  of  the  vast  amount  of  information,  having  separate  results  for  each  of  the 
items  in  the  assessment  pool  hinders  the  comparison  of  the  general  performance  of  subgroups  of  the 
population.  Item-by-item  reporting  masks  similarities  in  trends  and  subgroup  comparisons  that  are 
common  across  items. 

An  obvious  summary  of  performance  across  a collection  of  items  is  the  average  of  the  separate 
item  scores.  The  advantage  of  averaging  is  that  it  tends  to  cancel  out  the  effects  of  peculiarities  in  items 


' Nancy  L Allen  is  responsible  for  the  psychometric  and  statistical  analysis  of  national  and  state  NAEP  data.  Eugene  G.  Johnson 
is  a senior  psychometrician,  contributing  to  the  design  of  NAEP  and  to  discussions  of  sampling  issues.  Previously  he  was 
responsible  for  the  psychometric  and  statistical  analysis  of  NAEP  data.  Robert  J.  Mislevy  is  a technical  consultant  contributing  in 
the  area  of.item  response  theory.  Neal  Thomas  was. a technical  consultant  to  the  NAEP  analysis  staff,  contnbuting  in  the  area  of 
imputed  values. 
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that  can  affect  item  difficulty  in  unpredictable  ways.  Furthermore,  averaging  makes  it  possible  to 
compare  more  easily  the  general  performances  of  subpopulations. 


Despite  their  advantages,  there  are  a number  of  significant  problems  with  average  item  scores. 
First,  the  interpretation  of  these  results  depends  on  the  selection  of  the  items;  the  selection  of  easy  or 
difficult  items  could  make  student  performance  appear  to  be  overly  high  or  low.  Second,  the  average 
score  is  related  to  the  particular  items  comprising  the  average,  so  that  direct  comparisons  in  performance 
between  subpopulations  require  that  those  subpopulations  have  been  administered  the  same  set  of  items. 
Third,  because  this  approach  limits  comparisons  to  average  scores  on  specific  sets  of  items,  it  provides 
no  simple  way  to  report  trends  over  time  when  the  item  pool  changes.  Finally,  direct  estimates  of 
parameters  or  quantities  such  as  the  proportion  of  students  who  would  achieve  a certain  score  across  the 
items  in  the  pool  are  not  possible  when  every  student  is  administered  only  a fraction  of  the  item  pool. 
While  the  average  score  across  all  items  in  the  pool  can  be  readily  obtained  (as  the  average  of  the 
individual  item  scores),  statistics  that  provide  distributional  information,  such  as  quantiles  of  the 
distribution  of  scores  across  the  full  set  of  iterhs,  cannot  be  readily  obtained  without  additional 
assumptions. 

These  limitations  can  be  overcome  by  the  use  of  response  scaling  methods.  If  several  items 
require  similar  skills,  the  regularities  observed  in  response  patterns  can  often  be  exploited  to  characterize 
both  respondents  and  items  in  terms  of  a relatively  small  number  of  variables.  These  variables  include  a 
respondent-specific  variable,  called  proficiency,  which  quantifies  a respondent’s  tendency  to  answer 
items  correctly  (or,  for  multipoint  items,  to  achieve  a certain  score)  and  item-specific  variables  that 
indicate  characteristics  of  the  item  such  as  its  difficulty,  effectiveness  in  distinguishing  between 
individuals  with  different  levels  of  proficiency,  and  the  chances  of  a very  low  proficiency  respondent 
correctly  answering  a multiple-choice  item.  (These  variables  are  discussed  in  more  detail  in  the  next 
section.)  When  combined  through  appropriate  mathematical  formulas,  these  variables  capture  the 
dominant  features  of  the  data.  Furthermore,  all  students  can  be  placed  on  a common  scale,  even  though 
none  of  the  respondents  takes  all  of  the  items  within  the  pool.  Using  the  common  scale,  it  becomes 
possible  to  discuss  distributions  of  proficiency  in  a population  or  subpopulation  and  to  estimate  the 
relationships  between  proficiency  and  background  variables. 

It  is  important  to  point  out  that  any  procedure  of  aggregation,  from  a simple  average  to  a complex 
multidimensional  scaling  model,  highlights  certain  patterns  at  the  expense  of  other  potentially  interesting 
patterns  that  may  reside  within  the  data.  Every  item  in  a NAEP  survey  is  of  interest  and  can  provide 
useful  information  about  what  young  Americans  know  and  can  do.  The  choice  of  an  aggregation 
procedure  must  be  driven  by  a conception  of  just  which  patterns  are  salient  for  a particular  purpose. 

The  scaling  for  the  main  assessments  in  mathematics  and  science  was  carried  out  separately 
within  the  content  strands  or  fields  specified  in  the  frameworks  for  those  subjects,  respectively.  This 
scaling  within  subareas  was  done  because  it  was  anticipated  that  different  patterns  of  performance  or 
different  trends  over  time  might  exist  for  these  essential  subdivisions  of  the  subject  areas.  By  creating  a 
separate  scale  for  each  of  these  content  areas,  potential  differences  in  subpopulation  performance 
between  the  content  areas  are  preserved. 

The  creation  of  a series  of  separate  scales  to  describe  performance  within  a subject  area  does  not 
preclude  the  reporting  of  a single  index  of  overall  performance  in  the  subject  area — that  is,  an  overall 
subject  area  composite.  A composite  is  computed  as  the  weighted  average  of  the  content  area  scales, 
where  the  weights  correspond  to  the  relative  importance  given  to  each  content  area  as  defined  by  the 
framework.  The  composite  provides  a global  measure  of  performance  within  the  subject  area,  while  the 
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constituent  content  area  scales  allow  the  measurement  of  important  interactions  within  educationally 
relevant  subdivisions  of  the  subject  area. 


11.3  SCALING  METHODOLOGY 

This  section  reviews  the  scaling  models  employed  in  the  analyses  of  data  of  the  1996  assessment, 
and  the  multiple  imputation  or  “plausible  values”  methodology  that  allows  such  models  to  be  used  with 
NAEP’s  sparse  item-sampling  design.  The  reader  is  referred  to  Mislevy  (1991)  for  an  introduction  to 
plausible  values  methods  and  a comparison  with  standard  psychometric  analyses,  to  Donoghue  (1994), 
Mislevy,  Johnson  and  Muraki  (1992),  and  Beaton  and  Johnson  (1992)  for  additional  information  on  how 
the  models  are  used  in  NAEP,  and  to  Rubin  (1987)  for  the  theoretical  underpinnings  of  the  approach.  It 
should  be  noted  that  the  imputation  procedure  used  by  NAEP  is  a mechanism  for  providing  plausible 
values  for  the  unobserved  proficiencies  and  not  for  filling  in  blank  responses  to  background  or  cognitive 
variables. 

While  the  NAEP  procedures  were  developed  explicitly  to  handle  the  characteristics  of  NAEP 
data,  they  build  on  other  research,  and  are  paralleled  by  other  researchers.  See,  for  example  Dempster, 
Laird,  and  Rubin  (1977);  Little  and  Rubin  (1983,  1987);  Andersen  (1980);  Engelen  (1987);  Hoijtink 
(1991);  Laird  (1978);  Lindsey,  Clogg,  and  Grego  (1991);  Zwinderman  (1991);  Tanner  and  Wong  (1987); 
and  Rubin  (1987,  1991). 


11.3.1  The  Scaling  Models 

Three  distinct  scaling  models,  depending  on  item  type  and  scoring  procedure,  were  used  in  the 
analysis  of  the  data  from  the  1996  assessment.  Each  of  the  models  is  based  on  item  response  theory  (IRT; 
e.g..  Lord,  1980).  Each  is  a “latent  variable”  model,  defined  separately  for  each  of  the  scales,  which 
expresses  respondent’s  tendencies  to  achieve  certain  scores  (such  as  correct/incorrect)  on  the  items 
contributing  to  a scale  as  a function  of  a parameter  that  is  not  directly  observed,  called  proficiency  on  the 
scale. 


A three-parameter  logistic  (3PL)  model  was  used  for  the  multiple-choice  items  (which  were 
scored  correct/incorrect).  The  fundamental  equation  of  the  3PL  model  is  the  probability  that  a person 
whose  proficiency  on  scale  k is  characterized  by  the  unobservable  variable  0*  will  respond  correctly  to 
item  j: 


P(^Xj  l|^,  djj  Cj  -b 


1 + exp[-1.7aj(^i  - bj)] 


(11.1) 


where 

Xj  is  the  response  to  item  j,  1 if  correct  and  0 if  not; 

aj  where  aj>0,  is  the  slope  parameter  of  item  j,  characterizing  its  sensitivity  to 

proficiency; 

bj  is  the  threshold  parameter  of  item 7,  characterizing  its  difficulty;  and 

cj  where  is  the  lower  asymptote  parameter  of  item 7,  reflecting  the  chances  of 

students  of  very  low  proficiency  selecting  the  correct  option. 
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Further  define  the  probability  of  an  incorrect  response  to  the  item  as 

Pjo  = P{xj  = Q\6k,  aj,  bj,  Cj)  = 1 - Pj\{6k).  (11.2) 

A two-parameter  logistic  (2PL)  model  was  used  for  the  short  constructed-response  items  that 
were  scored  correct  or  incorrect.  The  form  of  the  2PL  model  is  the  same  as  Equations  (11.1)  and  (11.2) 
with  the  Cj  parameter  fixed  at  zero. 

In  addition  to  the  multiple-choice  and  short  constructed-response  items,  a number  of  extended 
constructed-response  items  were  presented  in  the  assessments  of  mathematics  and  science;  and  only 
extended  constructed-response  items  were  presented  in  the  long-term  trend  writing  assessment.  Each  of 
these  items  was  scored  on  a multipoint  scale  with  potential  scores  ranging  from  0 to  3 or  from  0 to  4. 
Some  short  constructed-response  items  were  scored  on  a three-point  scale  (0-2).  Items  that  are  scored  on 
a multipoint  scale  are  referred  to  as  polytomous  items,  in  contrast  with  the  multiple-choice  and  short 
constructed-response  items,  which  are  scored  correct  or  incorrect  and  referred  to  as  dichotomous  items. 


The  polytomous  items  were  scaled  using  a generalized  partial  credit  model  (Muraki,  1992).  The 
fundamental  equation  of  this  model  is  the  probability  that  a person  with  proficiency  0^  on  scale  k will 
have,  for  theyth  item,  a response  xj  that  is  scored  in  the  ith  of  ntj  ordered  score  categories! 


X - bj  -I-  dj,  v) 

p{xj  = tj0A:,  Uj,  bj,  dj,  I, . . . , dj,  mj  -])  = — j ^ = Pji(Qk)  (11-3) 

X exp  ^ 1. 7aX0*  - bj  -I-  dj,  v) 

g=Q  Vv=0  7 


where 


ntj  is  the  number  of  categories  in  the  response  to  item  j 

Xj  is  the  response  to  item  j,  with  possibilities  0,l,...,m,-l 

fly  is  the  slope  parameter; 

bj  is  the  item  location  parameter  characterizing  overall  difficulty;  and 
djj  is  the  category  i threshold  parameter  (see  below). 

Indeterminacies  in  the  parameters  of  the  above  model  are  resolved  by  setting  dj.o  = 0 and  setting 

my— 1 

X 4.  i = 0-  Muraki  (1992)  points  out  that  bj  - djj  is  the  point  on  the  0*  scale  at  which  the  plots  of  F,  , y(0k) 
1=1 

and  Fy,(0k)  intersect  and  so  characterizes  the  point  on  the  0^  scale  at  which  the  response  to  item  j has 
equal  probability  of  falling  in  response  category  i-1  and  falling  in  response  category  i. 

When  my  = 2,  so  that  there  are  two  score  categories  (0,1),  it  can  be  shown  that  Fy,(0t)  of  Equation 
(1 1.3)  for  i=0,l  corresponds  respectively  to  PjM  and  Fyy(0k)  of  the  2PL  model  (Equations  (11.1)  and 
(11.2)  with  cy=0). 
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Close  examination  of  the  3PL  and  generalized  partial  credit  models  indicate  that  both  models 
have  a linear  indeterminacy  of  the  theta  scale.  In  other  words,  if  the  item  parameters  were  estimated  in  a 
different  metric,  the  value  of  0*  could  be  transformed  to  make  (11.1)  and  (11.3)  true.  For  the  purposes  of 
reporting  item  parameter  estimates  and  other  intermediary  estimates,  the  linear  indeterminacies  apparent 
in  (1 1.1)  and  (11.3)  may  be  resolved  by  an  arbitrary  choice  of  the  origin  and  unit  size  in  a given  scale.  In 
most  cases,  a provisional  scale  standardizing  the  theta  distribution  to  have  mean  0 and  standard  deviation 
1 is  employed.  Final  results  for  each  content  area  were  linearly  transformed  from  the  0 scale  to  a 0-to- 
500  (for  mathematics)  or  a O-to-300  scale  (for  science),  as  described  in  the  subject  area  chapters  in  this 
report. 

A basic  assumption  of  item  response  theory  is  the  conditional  independence  of  the  responses  by 
an  individual  to  a set  of  items,  given  the  individual’s  proficiency.  That  is,  conditional  on  the 
individual’s  0jfc,  the  joint  probability  of  a particular  response  pattern  x = across  a set  of  n items 

is  simply  the  product  of  terms  based  on  (11.1),  (1 1.2),  and  (11.3): 

n mj-l 

P( x|0jfc,  item  parameters)  = f][  “ ( 1 1 -4) 

j=\  1=0 

where  Pji(Qk)  is  of  tho  form  appropriate  to  the  type  of  item  (dichotomous  or  polytomous),  mj  is  equal  to  2 
for  the  dichotomously  scored  items,  and  uj,  is  an  indicator  variable  defined  by 

{lif  response  Xj  is  in  category  i 
0 otherwise 

It  is  also  typically  assumed  that  response  probabilities  are  conditionally  independent  of 
background  variables  (y),  given  0jt,  or 

p(x|0jfc,  item  parameters,  y)  = p{x\Qk,  item  parameters).  (1 1 -5) 

After  X has  been  observed,  equation  11.4  can  be  viewed  as  a likelihood  function,  and  provides  a 
basis  for  inference  about  0jfc  or  about  item  parameters.  Estimates  of  item  parameters  were  obtained  by  the 
NAEP  BILOG/P ARSCALE  program,  which  combines  Mislevy  and  Bock’s  (1982)  BILOG  and  Muraki 
and  Bock’s  (1991)  PARSCALE  computer  programs^,  and  which  concurrently  estimates  parameters  for  all 
items  (dichotomous  and  polytomous).  Donoghue  (1994)  reports  on  the  effect  of  having  both  dichotomous 
and  polytomous  items  within  a scale.  The  item  parameters  are  then  treated  as  known  in  subsequent 
calculations.  In  subject  areas  with  multiple  scales  (main  mathematics  and  science),  the  parameters  of  the 
items  constituting  each  of  the  separate  scales  were  estimated  independently  of  the  parameters  of  the  other 
scales.  Once  items  have  been  calibrated  in  this  manner,  a likelihood  function  for  the  proficiency  0;^  is 
induced  by  a vector  of  responses  to  any  subset  of  calibrated  items,  thus  allowing  0<:-based  inferences 
from  matrix  samples.  The  likelihood  function  for  the  proficiency  0^  is  called  the  posterior  distribution  of 
the  thetas  for  each  student. 

In  almost  all  NAEP  IRT  analyses,  missing  responses  at  the  end  of  each  block  of  items  a student 
was  administered  were  considered  “not  reached,”  and  treated  as  if  they  had  not  been  presented  to  the 
respondent.  Missing  responses  to  dichotomous  items  before  the  last  observed  response  in  a block  were 
considered  intentional  omissions,  and  treated  as  fractionally  correct  at  the  value  of  the  reciprocal  of  the 


^ See  Muraki  and  Bock  (1997)  for  the  current  version  of  PARSCALE. 
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number  of  response  alternatives,  if  the  item  was  a multiple-choice  item.  These  conventions  are  discussed 
by  Mislevy  and  Wu  (1988).  With  regard  to  the  handling  of  not-reached  items,  Mislevy  and  Wu  found  that 
ignoring  not-reached  items  introduces  slight  biases  into  item  parameter  estimation  when  not-reached 
items  are  present  and  speed  is  correlated  with  ability.  With  regard  to  omissions,  they  found  that  the 
method  described  above  provides  consistent  limited-information  maximum  likelihood  estimates  of  item 
and  ability  parameters  under  the  assumption  that  respondents  omit  only  if  they  can  do  no  better  than 
responding  randomly. 

Occasionally,  extended  constructed-response  items  were  the  last  item  in  a block  of  items. 

Because  considerably  more  effort  was  required  of  the  student  to  answer  these  items,  nonresponse  to  an 
extended  constructed-response  item  at  the  end  of  a block  was  considered  an  intentional  omission  (and 
scored  as  the  lowest  category,  0)  unless  the  student  also  did  not  respond  to  the  item  immediately 
preceding  that  item.  In  that  case,  the  extended  constructed-response  item  was  considered  not  reached  and 
treated  as  if  it  had  not  been  presented  to  the  student. 

Although  the  IRT  models  are  employed  in  NAEP  only  to  summarize  performance,  a number  of 
checks  are  made  to  detect  serious  violations  of  the  assumptions  underlying  the  models.  Checks  are  made 
to  detect  multidimensionality  of  the  construct  being  measured  and  certain  condition  dependencies.  DIF 
analyses  are  used  to  examine  issues  of  dimensionality,  and  what  are  called  statistics  in  the  IRT 
literature  are  used  to  flag  responses  with  serious  departures  from  the  IRT  model.  The  latter  statistics 
might  better  be  called  item  fit  statistics  since  they  do  not  really  have  distributions.  These  checks 
include  comparisons  of  empirical  and  theoretical  item  response  functions  to  identify  items  for  which  the 
IRT  model  may  provide  a poor  fit  to  the  data.  When  warranted,  remedial  efforts,  such  as  collapsing 
categories  of  polytomous  items  or  combining  items  into  a single  item,  are  made  to  mitigate  the  effects  of 
such  violations  on  inferences. 

Scaling  areas  in  NAEP  are  determined  a priori  by  grouping  items  into  content  areas  for  which 
overall  performance  is  deemed  to  be  of  interest,  as  defined  by  the  frameworks  developed  by  the  National 
Assessment  Governing  Board  (NAGB).  A proficiency  scale  0*  is  defined  a priori  by  the  collection  of 
items  representing  that  scale.  What  is  important,  therefore,  is  that  the  models  capture  salient  information 
in  the  response  data  to  effectively  summarize  the  overall  performance  on  the  content  area  of  the 
populations  and  subpopulations  being  assessed  in  the  content  areas.  NAEP  has  routinely  conducted 
differential  item  functioning  (DIF)  analyses  to  guard  against  potential  biases  in  making  subpopulation 
comparisons  based  on  the  proficiency  distributions. 

The  local  independence  assumption  embodied  in  Equation  (11.4)  implies  that  item  response 
probabilities  depend  only  on  0 and  the  specified  item  parameters,  and  not  on  the  position  of  the  item  in 
the  booklet,  the  content  of  items  around  an  item  of  interest,  or  the  test-administration  and  timing 
conditions.  However,  these  effects  are  certainly  present  in  any  application.  The  practical  question  is 
whether  inferences  concerning  aggregate  performance  in  the  scaling  area  that  are  based  on  the  IRT 
probabilities  obtained  via  (11.4)  are  robust  with  respect  to  the  ideal  assumptions  underlying  the  IRT 
model.  Our  experience  with  the  1986  NAEP  reading  anomaly  (Beaton  & Zwick,  1990)  has  shown  that  for 
measuring  small  changes  over  time,  changes  in  item  context  and  speededness  conditions  can  lead  to 
unacceptably  large  random  error  components.  These  can  be  avoided  by  presenting  items  used  to  measure 
change  in  identical  test  forms,  with  identical  timings  and  administration  conditions.  Thus,  we  do  not 
maintain  that  the  item  parameter  estimates  obtained  in  any  particular  booklet  configuration  are 
appropriate  for  other  conceivable  configurations.  Rather,  we  assume  that  the  parameter  estimates  are 
context-bound.  This  is  the  reason  that  the  long-term  trend  booklets  and  administration  procedures  have 
not  changed  since  the  early  1980s  and  only  a limited  number  of  blocks  of  items  are  released  after  each 
main  assessment  cycle.  It  was  also  the  reason  we  prefer  common  population  equating  to  common  item 
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equating  whenever  equivalent  random  samples  are  available  for  linking.  In  common  item  equating,  items 
are  assumed  to  be  measuring  exactly  the  same  thing  for  two  or  more  populations,  despite  any  differences 
in  context  or  administration.  In  common  population  equating,  results  for  two  or  more  samples  from  the 
same  population  are  matched  to  one  another  when  linking  the  scales.  Therefore,  the  data  from  the  State 
Assessment  were  calibrated  separately  from  the  national  NAEP  data.  In  this  case,  the  administration 
procedures  differed  somewhat  between  the  State  Assessment  and  the  national  NAEP. 

In  practice,  PARSCALE  item  fit  statistics  are  used  as  a way  to  identify  items  that  need  further 
examination.  Most  of  the  statistics  of  this  type  that  are  available  for  use  in  this  setting  have  distributions 
that  are  unknown.  Therefore,  they  cannot  be  used  for  final  decisions  about  the  fit  of  the  items  to  the  IRT 
model.  Because  of  the  lack  of  statistical  tests  for  IRT  model  fit,  the  fit  of  the  IRT  models  to  the  observed 
data  was  examined  within  each  scale  by  comparing  the  empirical  item  response  functions  (IRFs)  with  the 
theoretical  curves.  The  primary  means  of  accomplishing  this  is  to  generate  plots  of  empirical  versus 
theoretical  item  response  curves.  The  theoretical  curves  are  plots  of  the  response  functions  based  on  the 
estimates  of  the  item  parameters.  The  empirical  proportions  are  calculated  from  the  posterior 
distributions  of  the  thetas  for  each  student  who  received  the  item.  For  dichotomous  items,  the  sum  of  the 
values  of  the  posterior  distributions  at  a point  on  the  theta  scale  for  each  student  who  answered  an  item 
correctly  plus  the  sum  of  a fractional  portion  of  the  values  of  the  posterior  distribution  at  that  point  on  the 
theta  scale  for  each  student  who  omitted  the  item  is  parallel  in  meaning  to  the  number  of  students  who 
actually  answered  the  item  correctly  plus  a fraction  of  the  number  of  students  who  omitted  the  item.  The 
sum  of  the  values  of  the  posterior  distributions  for  all  students  receiving  the  item  at  each  point  on  the 
theta  scale  is  parallel  in  meaning  to  the  empirical  number  of  students  at  that  point  on  the  theta  scale  who 
received  the  item.  The  plotted  values  are  sums  of  these  individual  posteriors  at  each  point  on  the  theta 
scale  for  those  who  got  the  item  correct  plus  a fraction  of  the  omitters  divided  by  the  sum  of  the 
posteriors  of  those  administered  the  item,  in  the  case  of  dichotomous  items,  and  for  those  who  scored  in 
the  category  of  interest  over  the  sum  for  those  who  received  the  item,  in  the  case  of  polytomous  items. 

Figure  1 1-1  contains  a plot  of  the  empirical  and  theoretical  IRFs  for  a dichotomous  item.  In  the 
plot,  the  horizontal  axis  represents  the  theta  (proficiency)  scale,  the  vertical  axis  represents  the 
probability  of  a correct  response.  The  solid  curve  is  the  theoretical  IRF  based  on  the  item  parameter 
estimates  and  Equation  (11.1).  The  centers  of  the  diamonds  represent  the  empirical  proportions  correct  as 
described  above.  The  size  of  the  diamonds  are  proportional  to  the  sum  of  the  posteriors  at  each  point  on 
the  theta  scale  for  all  of  those  who  received  the  item;  this  is  related  to  the  number  of  students 
contributing  to  the  estimation  of  that  empirical  proportion  correct. 

Figure  11-2  contains  a plot  of  the  empirical  and  theoretical  IRFs  for  a polytomous  item  from  the 
1997  Arts  (Theatre)  National  Assessment.  As  for  the  dichotomous  item  plot  in  Figure  1 1-1,  the 
horizontal  axis  represents  the  proficiency  scale,  but  the  vertical  axis  represents  the  probability  of  having 
a response  fall  in  each  category.  The  solid  curves  are  the  theoretical  IRFs  based  on  the  item  parameter 
estimates  and  Equation  (11.3).  The  centers  of  the  diamonds  represent  the  empirical  proportions  of 
students  with  responses  in  each  category  and  are  proportional  to  the  sum  of  the  posteriors  at  each  point 
on  the  theta  scale  for  the  students  who  received  the  item. 

For  good  fitting  items,  the  empirical  and  theoretical  curves  are  close  together.  Therefore,  items 
for  which  this  is  not  true  are  examined  carefully.  Examples  of  plots  for  specific  items  are  provided  in  the 
subject-area  chapters.  When  the  same  items  are  presented  in  two  assessment  years,  the  empirical  curves 
for  the  two  years  can  be  compared.  Normally,  these  curves  differ  somewhat  due  to  the  sampling  of 
students  for  each  of  the  two  years.  Figure  12-1  contains  a plot  for  an  item  with  curves  of  this  type.  When 
the  empirical  curves  differ  dramatically,  one  cause  might  be  a change  in  the  meaning  of  the  item  due  to 
instructional  or  societal  changes  across  the  years.  This  type  of  item  is  ordinarily  treated  as  two  different 
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items— one  for  each  of  the  assessment  years.  Figure  12-4  contains  the  plot  for  an  item  that  has  been 
treated  in  this  way. 


To  summarize,  using  current  methodologies  in  psychometrics,  assumption  of  conditional 
independence  and  the  assumption  that  the  data  fit  the  models  in  Equations  1 1.1  and  1 1.3  are  examined 
and  controlled  in  NAEP  in  several  ways.  They  are  examined  by  considering  tests  of  DIF,  item  fit 
statistics,  and  plots  of  empirical  and  theoretical  IRFs.  They  are  controlled  by  treating  mining  and  “not 
reached”  responses  in  reasonable  ways,  maintaining  the  context  and  administration  of  items  across 
assessments,  collapsing  categories  of  polytomous  items  when  appropriate,  combining  items  into  a single 
item,  or  making  decisions  about  the  inclusion  or  exclusion  of  an  item  in  a scale  based  on  data.  The 
identification  and  amelioration  of  violations  of  IRT  assumptions  is  an  area  of  ongoing  research  in 
educational  measurement. 


Figure  11-1 

Example  Cross-Sectional  Dichotomous  Item  (R016102,  Age  13/Grade  8) 
Exhibiting  Good  Model  Fit* 


R 0 1 6 1 0 2 


u . u 

THETA 


^Notc,  The  plot  compares  empirical  and  model-based  estimates  of  the  item  response  function 
(IRF),  The  smooth  curve  represents  the  model-based  estimate  at  each  provisional  proficiency 
level  The  diamonds  represent  the  empirical  proportion  of  1994  age  13/grade  8 students 
answering  correctly  at  each  point  on  the  theta  scale. 


Figure  11-2 

Example  Polytomous  Item  (HC00004,  Grade  8)  Ejchibiting  Good  Model  Fit* 


NAEPIO:  HC00004  SEQUENCE:  004  SCALE:  THEATRE 


*Note:  The  plot  compares  empirical  and  model-based  estimates  of  the  item  category 
response  functions  (ICRFs).  The  smooth  curve  represents  the  model-based  estimate  at 
each  provisional  proficiency  level.  The  diamonds  represent  the  empirical  proportion  of 
1997  grade  8 students  with  responses  in  each  categorydt  each  point  on  the  theta  scale. 


11.3.2  An  Overview  of  Plausible  Values  Methodology 

Item  response  theory  was  developed  in  the  context  of  measuring  individual  examinees’  abilities. 

In  that  setting,  each  individual  is  administered  enough  items  (often  60  or  more)  to  permit  precise 
estimation  of  his  or  her  0,  as  a maximum  likelihood  estimate,  6 , for  example.  Because  the  uncertainty 
associated  with  each  0 is  negligible,  the  distribution  of  0,  or  the  joint  distribution  of  0 with  other 
variables,  can  then  be  approximated  using  an  individual’s  0 values  as  if  they  were  0 values. 

This  approach  breaks  down  in  the  assessment  setting  when,  in  order  to  provide  broader  content 
coverage  in  limited  testing  time,  each  respondent  is  administered  relatively  few  items  in  a subject  area 
scale.  A first  problem  is  that  the  uncertainty  associated  with  individual  0s  is  too  large  to  ignore,  and  the 
features  of  the  0 distribution  ctin  be  seriously  biased  as  estimates  of  the  0 distribution.  (The  failure  of 
this  approach  was  verified  in  early  analyses  of  the  1984  NAEP  reading  survey;  see  Wingersky,  Kaplan, 

& Beaton,  1987.)  A second  problem,  occurring  even  with  test  lengths  of  60,  arises  when  test  forms  vary 
across  and  within  assessments  as  to  the  numbers,  formats,  and  content  of  the  test  items.  The  measurement 
error  distributions  thus  differ  even  if  underlying  0 distributions  do  not,  causing  0 distributions  to  exhibit 
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spurious  changes  and  comparisons  in  apparent  population  distributions — easily  greater  than  actual 
differences  over  time  or  across  groups.  Although  this  latter  problem  is  avoided  in  traditional  standardized 
testing  by  presenting  students  with  parallel  test  forms,  controlled  tightly  across  time  and  groups,  the  same 
constraints  cannot  be  imposed  in  the  design  and  data-collection  phases  of  the  present  NAEP.  Plausible 
values  were  developed  as  a way  to  estimate  key  population  features  consistently,  and  approximate  others 
no  worse  than  standard  IRT  procedures  would,  even  when  item  booklet  composition,  format,  and  content 
balances  change  over  time.  A detailed  development  of  plausible  values  methodology  is  given  in  Mislevy 
(1991).  Along  with  theoretical  justifications,  that  paper  presents  comparisons  with  standard  procedures, 
discussions  of  biases  that  arise  in  some  secondary  analyses,  and  numerical  exarhples. 

The  following  provides  a brief  overview  of  the  plausible  values  approach,  focusing  on  its 
implementation  in  the  1996  NAEP  analyses. 

Let  X represent  the  responses  of  all  sampled  examinees  to  background  and  attitude  questions, 
along  with  variables  based  on  the  sampling  design  such  as  the  school  where  the  student  is  enrolled,  and 
let  0 represent  the  vector  of  proficiency  values.  If  0 were  known  for  all  sampled  examinees,  it  would  be 
possible  to  compute  a statistic  t(0,y),  such  as  a scale  or  composite  subpopulation  sample  mean,  a sample 
percentile  point,  or  a sample  regression  coefficient,  to  estimate  a corresponding  population  quantity  T.  A 
function  t/(0,y)— e.g.,  a jackknife  estimate— would  be  used  to  gauge  sampling  uncertainty,  as  the 
variance  of  t around  T in  repeated  samples  from  the  population. 

Because  the  scaling  models  are  latent  variable  models,  however,  0 values  are  not  observed  even 
for  sampled  students.  To  overcome  this  problem,  we  follow  Rubin  (1987)  by  considering  0 as  “missing 

data,”  and  approximate  r(0,y)  by  its  expectation  given  fey),  the  data  that  actually  were  observed,  as 
follows: 


It  IS  possible  to  approximate  t*  using  random  draws  from  the  predictive  conditional  distribution 
of  the  scale  proficiencies  given  the  item  responses  background  variables  y„  and  model  parameters  for 
sampled  student  i.  These  values  are  referred  to  as  imputations  in  the  sampling  literature,  and  plausible 
values  in  NAEP.  The  value  of  0 for  any  respondent  that  would  enter  into  the  computation  of  t is  thus 
replaced  by  a randomly  selected  value  from  the  respondent’s  conditional  distribution.  Rubin  (1987) 
proposes  that  this  process  be  carried  out  several  times — multiple  imputations — so  that  the  uncertainty 
associated  with  imputation  can  be  quantified.  The  average  of  the  results  of,  for  example,  M estimates  of  t, 
each  computed  from  a different  set  of  plausible  values,  is  a Monte  Carlo  approximation  of  (1 1 .6);  the 
variance  among  them,  B,  reflects  uncertainty  due  to  not  observing  0,  and  must  be  added  to  the  estimated 
expectation  of  t/(0,y),  which  reflects  uncertainty  due  to  testing  only  a sample  of  students  from  the 
population.  Section  1 1.5  explains  how  plausible  values  are  used  in  subsequent  analyses. 

It  cannot  be  emphasized  too  strongly  that  plausible  values  are  not  test  scores  for  individuals  in 
the  usual  sense.  Plausible  values  are  offered  only  as  intermediary  computations  for  calculating  integrals 
of  the  form  of  Equation  ( 1 1 .6),  in  order  to  estimate  population  characteristics.  When  the  underlying 
model  is  correctly  specified,  plausible  values  will  provide  consistent  estimates  of  population 
characteristics,  even  though  they  are  not  generally  unbiased  estimates  of  the  proficiencies  of  the 
individuals  with  whom  they  are  associated.  The  key  idea  lies  in  the  contrast  between  plausible  values  and 


the  more  familiar  estimates  of  proficiency  (e.g.,  maximum  likelihood  estimate  or  Bayes  estimate)  that  are 
in  some  sense  optimal  for  each  examinee:  Point  estimates  that  are  optimal  for  individual  examinees  have 
distributions  that  can  produce  decidedly  nonoptimal  (specifically,  inconsistent)  estimates  of  population 
characteristics  (Little  & Rubin,  1983).  Plausible  values,  on  the  other  hand,  are  constructed  explicitly  to 
provide  consisterit  estimates  of  population  effects.  For  further  discussion  see  Mislevy,  Beaton,  Kaplan, 
and  Sheehan  (1992). 


11,3,3  Computing  Plausible  Values  in  IRT-Based  Scales 

Plausible  values  for  each  respondent  r are  drawn  from  the  predictive  conditional  distribution 
p(Qr\^yr,r,U),  where  /’and  27 are  regression  model  parameters  defined  in  this  subsection.  This 
subsection  describes  how,  in  IRT-based  scales,  these  conditional  distributions  are  characterized,  and  how 
the  draws  are  taken.  An  application  of  Bayes’  theorem  with  the  IRT  assumption  of  conditional 
independence  produces 


i^)p(0\yr,  r,i:)  = p{^\0r)p(0\^,r,  27)  (11.7) 


where,  for  vector-valued  0„  P(Xr\Qr)  is  the  product  over  scales  of  the  independent  likelihoods  induced  by 
responses  to  items  within  each  scale,  and  p(Qr\yr,r,U)  is  the  multivariate— and  generally 
nonindependent — joint  density  of  proficiencies  for  the  scales,  conditional  on  the  observed  value  of 
background  responses,  and  the  parameters  /’and  27.  The  provisional  scales  are  determined  by  the  item 
parameter  estimates  that  constrain  the  population  mean  to  zero  and  standard  deviation  to  one.  The  item 
parameter  estimates  are  fixed  and  regarded  as  population  values  in  the  computation  described  in  this 
subsection. 


In  the  analyses  of  the  data  from  the  main  assessments,  a normal  (Gaussian)  form  was  assumed  for 
p(Qr\yr,r,I^,  with  a common  variance-covariance  matrix,  27,  and  with  a mean  given  by  a linear  model 
with  slope  parameters,  F,  based  on  the  first  approximately  200  principal  components  of  several  hundred 
selected  main-effects  and  two-way  interactions  of  the  complete  vector  of  background  variables.  The 
included  principal  components  will  be  referred  to  as  the  conditioning  variables,  and  will  be  denoted  yf . 
(The  complete  set  of  original  background  variables  used  in  the  analyses  of  each  subject  area  are  listed  in 
Appendix  C.)  The  following  model  was  fit  to  the  data  within  each  subject  area: 

0=r’/  + e (11-8) 

where  e is  multivariately  normally  distributed  with  mean  zero  and  variance-covariance  matrix  27.  The 
number  of  principal  components  of  the  background  variables  used  for  each  sample  was  sufficient  to 
account  for  90  percent  of  the  total  variance  of  the  full  set  of  background  variables  (after  standardizing 
each  variable).  As  in  regression  analysis,  Fis  a matrix  each  of  whose  columns  is  the  effects  for  one  scale 
and  27  is  the  matrix  variance-covariance  of  residuals  between  scales. 

A model  similar  to  (1 1.8)  was  used  for  the  long-term  trend  assessments,  with  the  difference  that 
yf  consisted  of  main  effects  and  interactions  from  the  smaller  set  of  background  variables  (rather  than 
principal  components  of  those  variables)  available  in  the  long-term  trend  assessments. 
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Maximum  likelihood  estimates  of  /"and  S,  denoted  by  T and  I" , are  obtained  with  extensions 
of  Sheehan’s  (1985)  MGROUP  computer  program  using  the  EM  algorithm  described  in  Mislevy  (1985) 
The  EM  algorithm  requires  the  computation  of  the  mean,  0r,  and  variance-covariance  matrix,  ^ f , of  the 
predictive  conditional  distribution  in  (11.7)  for  respondent  r when  there  are  p scales  within  a subject 
area.  For  subject  areas  with  multiple  scales,  the  CGROUP  version  of  the  MGROUP  program  was  used  to 
compute  the  moments  using  higher  order  asymptotic  corrections  to  a normal  approximation  (Thomas, 
1993).  For  the  long-term  trend  assessments,  each  of  which  have  a single  scale,  the  more  precise  but 
computationally  intensive  BGROUP  version  of  MGROUP  was  used.  BGROUP  uses  numeric  quadrature 
to  evaluate  the  predictive  conditional  distribution  moments  required  by  the  E-step  of  the  EM  algorithm 
for  one-  and  two-dimensional  applications  (Thomas,  1993).  For  estimation  of  group  means  on  a single 
scale,  CGROUP  and  BGROUP  results  will  be  nearly  identical  to  those  from  the  original  MGROUP 
program.  CGROUP  and  BGROUP  yield  better  estimates  of  correlations  between  scales,  and  hence  better 
estimates  of  composite  scale  means.  BGROUP  will,  theoretically,  yield  better  estimates  than  CGROUP, 
but  because  of  the  heavy  computational  demands  of  the  methodology  used,  its  function  is  limited  to 
bivariate  scales.  Hence  CGROUP  is  used  for  assessments  involving  more  than  two  scales. 

After  completion  of  the  EM  algorithm,  the  plausible  values  for  all  sampled  respondents  is  drawn 
in  the  following  three-step  process.  First,  a value  of  /"is  drawn  from  a normal  distribution  with  mean 

^ yv 

being  /"  and  variance  being  the  variance  of  /".  Second,  conditional  on  the  generated  value  of  /"and  the 

fixed  value  of  2j,  the  predictive  conditional  distribution  mean  0r  and  the  predictive  conditional 
distribution  variance  I , of  respondent  r are  computed  from  Equation  1 1.7  using  the  EM  algorithm  (see 
Thomas,  1993).  Finally,  the^  are  drawn  independently  from  a multivariate  normal  distribution  with 

mean  ^ and  variance  approximating  the  distribution  in  (11.7).  These  three  steps  are  repeated  five 
times  producing  five  sets  of  imputation  values  for  all  sampled  respondents. 


11.4  INFERENCES  ABOUT  PROFICIENCIES 


When  survey  variables  are  observed  without  error  from  every  respondent,  usual  variance 
estimators  quantify  the  uncertainty  associated  with  sample  statistics  from  the  only  source  of  uncertainty, 
namely  the  sampling  of  respondents.  Item-level  statistics  for  NAEP  cognitive  items  meet  this 
requirement,  but  proficiency  values  do  not.  The  IRT  models  used  in  their  construction  posit  an 
unobservable  proficiency  variable  0 to  summarize  performance  on  the  items  in  the  subarea.  The  fact  that 
0 values  are  not  observed  even  for  the  respondents  in  the  sample  requires  additional  statistical  analyses 
to  draw  inferences  about  0 distributions  and  to  quantify  the  uncertainty  associated  with  those  inferences. 
As  described  above,  Rubin’s  (1987)  multiple  imputations  procedures  were  adapted  to  the  context  of 
latent  variable  models  to  produce  the  plausible  values  upon  which  many  analyses  of  the  data  from  the 
1996  assessment  were  based.  This  section  describes  how  plausible  values  were  employed  in  subsequent 
analyses  to  yield  inferences  about  population  and  subpopulation  distributions  of  proficiencies. 


11.4.1  Computational  Procedures 

Even  though  one  does  not  observe  the  0 value  of  respondent  r,  one  does  observe  variables  that 
are  related  to  it:  Xr,  the  respondent’s  answers  to  the  cognitive  items  he  or  she  was  administered  in  the  area 
of  interest,  and  the  respondent’s  answers  to  demographic  and  background  variables.  Suppose  one 
wishes  to  draw  inferences  about  a number  7(0,  F)  that  could  be  calculated  explicitly  if  the  0 and  y values 
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of  each  member  of  the  population  were  known.  Suppose  further  that  if  0 values  were  observable,  we 
would  be  able  to  estimate  T from  a sample  of  N pairs  of  0 and  y values  by  the  statistic  r(0,yj  [where 
(Q^y)  = (d,,y,....,ds.yN)l  and  that  we  could  estimate  the  variance  in  t around  T due  to  sampling 
r^pondents  by  the  function  U(Q,y).  Given  that  observations  consist  of  (xr^r)  rather  than  (Qr,yr),  we  can 
approximate  t by  its  expected  value  conditional  on  (^y),  or 

t*  (^y)  = E[t(d,y)\x,y]  = /t(Q,y)  p(d  lx,yj  dQ  . 

It  is  possible  to  approximate  t*  with  random  draws  from  the  conditional  distributions  p(0ilx,-,yj, 
which  are  obtained  for  all  respondents  by  the  method  described  in  Section  1 1 .3.3.  Let  0m  be  the  mth  such 
vector  of  plausible  values,  consisting  of  a multidimensional  value  for  the  latent  variable  of  each 
respondent.  This  vector  is  a plausible  representation  of  what  the  true  0 vector  might  have  been,  had  we 
been  able  to  observe  it. 

The  following  steps  describe  how  an  estimate  of  a scalar  statistic  t(d,y)  and  its  sampling  variance 
can  be  obtained  from  M (>1)  such  sets  of  plausible  values.  (Five  sets  of  plausible  values  are  used  in 
NAEP  analyses.) 

1 . Using  each  set  of  plausible  values  0m  in  turn,  evaluate  t as  if  the  plausible  values 
were  true  values  of  0.  Denote  the  results  t„,  for 

2.  Using  the  jackknife  variance  estimator  defined  in  Chapter  10,  compute  the  estimated 
sampling  variance  of  tm,  denoting  the  result  Um- 

3.  The  final  estimate  of  r is 


t* 


4.  Compute  the  average  sampling  variance  over  the  M sets  of  plausible  values,  to 
approximate  uncertainty  due  to  sampling  respondents 


M TJ 

u*=Y— 


5.  Compute  the  variance  among  the  M estimates  ?m,  to  approximate  the  between- 
imputation  variance 


h (w-i)  ' 


6.  The  final  estimate  of  the  variance  of  / is  the  sum  of  two  components 


V = U*+{l  + M~')B. 


In  this  equation,  (1+AT')B  is  the  estimate  of  variance  due  to  the  latency  of  0.  Due  to  the 
) excessive  computation  that  would  be  required,  NAEP  analyses  did  not  compute  and 
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average  jackknife  variances  over  all  five  sets  of  plausible  values,  but  only  on  the  first  set. 
Thus,  in  NAEP  reports,  U*  is  approximated  by  Uj. 


11.4.2  Statistical  Tests 

The  variance  described  in  Section  11.4.1  is  used  to  make  statistical  tests  comparing  NAEP 
results.  This  section  describes  the  relationships  between  these  tests  and  the  variance  components 
described  above.  Chapter  18  contains  details  of  the  hypothesis  tests  used  in  this  assessment. 

Suppose  that  if  0 values  were  observed  for  sampled  students,  the  statistic  (t  - T)/U'^  would 
follow  a t-distribution  with  d degrees  of  freedom.  Then  the  incomplete-data  statistic  (t*  - T)/V^^  is 
approximately  r-distributed,  with  degrees  of  freedom  (Satterthwaite,  1941;  Johnson  & Rust,  1993) 
given  by 

""  f ,(l-/r 

M-1  d 

where /is  the  proportion  of  total  variance  due  to  not  observing  0 values: 

/ ={l  + M-')B/V. 


When  B is  small  relative  to  f/*,  the  reference  distribution  for  incomplete-data  statistics  differs 
little  from  the  reference  distribution  for  the  corresponding  complete-data  statistics.  This  is  the  case  with 
main  NAEP  reporting  variables.  If,  in  addition,  d is  large,  the  normal  approximation  can  be  used  to  flag 
“significant”  results. 

For  k-dimensional  t,  such  as  the  k coefficients  in  a multiple  regression  analysis,  each  11^  and  U* 
is  a covariance  matrix,  and  B is  an  average  of  squares  and  cross-products  rather  than  simply  an  average  of 
squares.  In  this  case,  the  quantity  (T-{)  V'  (T-t),  is  approximately  F distributed,  with  degrees  of 
freedom  equal  to  k and  v,  with  v defined  as  above  but  with  a matrix  generalization  of/ 


/=  Trace  (BV')/k  . 

By  the  same  reasoning  as  used  for  the  normal  approximation  for  scalar  t,  a chi-square  distribution  on  k 
degrees  of  freedom  often  suffices  for  multivariate  t. 


11.4.3  Biases  in  Secondary  Analyses 


Statistics  r*  that  involve  proficiencies  in  a scaled  content  area  and  variables  included  in  the 
conditioning  variables  are  consistent  estimates  of  the  corresponding  population  values  T.  This  includes 
interrelationships  among  scales  within  a content  area  that  have  been  treated  in  the  multivariate  manner 
described  above  in  Section  1 1.3.3.  Statistics  involving  background  variables  y that  were  not  conditioned 
on,  or  relationships  among  proficiencies  from  different  content  strands  or  fields,  are  subject  to  asymptotic 
biases  whose  magnitudes  depend  on  the  type  of  statistic  and  the  strength  of  the  relationships  of  the 
nonconditioned  background  variables  to  the  variables  that  were  conditioned  on  and  to  the  proficiency  of 
interest.  That  is,  the  large  sample  expectations  of  certain  sample  statistics  need  not  equal  the  true 
population  parameters. 
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The  direction  of  the  bias  is  typically  to  underestimate  the  effect  of  nonconditioned  variables.  For 
details  and  derivations  see  Beaton  and  Johnson  (1990),  Mislevy  (1991),  and  Mislevy  and  Sheehan  (1987, 
Section  10.3.5).  For  a given  statistic  t*  involving  one  content  area  and  one  or  more  nonconditioned 
background  variables,  the  magnitude  of  the  bias  is  related  to  the  extent  to  which  observed  responses  x 
account  for  the  latent  variable  6,  and  the  degree  to  which  the  nonconditioned  background  variables  are 
explained  by  conditioning  background  variables.  The  first  factor — conceptually  related  to  test 
reliability — acts  consistently  in  that  greater  measurement  precision  reduces  biases  in  all  secondary 
analyses.  The  second  factor  acts  to  reduce  biases  in  certain  analyses  but  increase  it  in  others.  In 
particular, 

• High  shared  variance  between  conditioned  and  nonconditioned  background  variables 
mitigates  biases  in  analyses  that  involve  only  proficiency  and  nonconditioned 
variables,  such  as  marginal  means  or  regressions. 

• High  shared  variance  exacerbates  biases  in  regression  coefficients  of  conditional 
effects  for  nonconditioned  variables,  when  nonconditioned  and  conditioned 
background  variables  are  analyzed  jointly  as  in  multiple  regression. 

The  large  number  of  background  variables  that  have  been  included  in  the  conditioning  vectors  for  the 
1996  assessments  allows  a large  number  of  secondary  analyses  to  be  carried  out  with  little  or  no  bias,  and 
mitigates  biases  in  analyses  of  the  marginal  distributions  of  0 in  nonconditioned  variables.  Kaplan  and 
Nelson’s  analysis  of  the  1988  NAEP  reading  data  (some  results  of  which  are  summarized  in  Mislevy, 
1991),  which  had  a similar  design  and  fewer  conditioning  variables,  indicates  that  the  potential  bias  for 
nonconditioned  variables  in  multiple  regression  analyses  is  below  10  percent,  and  biases  in  simple 
regression  of  such  variables  is  below  five  percent.  Additional  research  (summarized  in  Mislevy,  1990) 
indicates  that  most  of  the  bias  reduction  obtainable  from  conditioning  on  a large  number  of  variables  can 
be  captured  by  instead  conditioning  on  the  first  several  principal  components  of  the  matrix  of  all  original 
conditioning  variables.  This  procedure  was  adopted  for  the  1992, 1994,  and  1996  main  assessments  by 
replacing  the  conditioning  effects  by  the  first  K principal  components,  where  K was  selected  so  that  90 
percent  of  the  total  variance  of  the  full  set  of  conditioning  variables  (after  standardization)  was  captured. 
Mislevy  (1990)  shows  that  this  puts  an  upper  bound  of  10  percent  on  the  average  bias  for  all  analyses 
involving  the  original  conditioning  variables. 


11.4.4  A Numerical  Example 

To  illustrate  how  plausible  values  are  used  in  subsequent  analyses,  this  subsection  gives  some  of  the 
steps  in  the  calculation  of  the  1992  grade  4 reading  composite  mean  and  its  estimation-error  variance. 
This  illustration  is  an  example  of  the  calculation  of  NAEP  means  and  variances  and  can  be  used  to 
understand  their  calculation  for  any  NAEP  assessment. 

The  weighted  mean  of  the  first  plausible  values  of  the  reading  composite  for  the  grade  4 students 
in  the  sample  is  217.79,  and  the  jackknife  variance  of  these  values  is  0.833.  Were  these  values  true  0 
values,  then  217.79  would  be  the  estimate  of  the  mean  and  0.833  would  be  the  estimation-error  variance. 
The  weighted  mean  of  the  second  plausible  values  of  the  same  students,  however,  is  217.62;  the  third, 
fourth,  and  fifth  plausible  values  give  weighted  means  of  217.74,  218.24,  and  218.05.  Since  all  of  these 
figures  are  based  on  precisely  the  same  sample  of  students,  the  variation  among  them  is  due  to 
uncertainty  about  the  students’  0s,  having  observed  their  item  responses  and  background  variables. 
Consequently,  our  best  estimate  of  the  mean  for  grade  4 students  is  the  average  of  the  five  plausible 
values:  217.89.  Taking  the  jackknife  variance  estimate  from  the  first  plausible  value,  0.833,  as  our 
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estimate  U*  of  sampling  variance,  and  the  variance  among  the  five  weighted  means,  .063,  as  our  estimate 
B of  uncertainty  due  to  not  observing  0,  we  obtain  as  the  final  estimate  V of  total  error  variance  0 833  + 
(1+5  ')  .063  = 0.908. 

It  is  also  possible  to  partition  the  estimation  error  variance  of  a statistic  using  these  same 
variance  components.  The  proportion  of  error  variance  due  to  sampling  students  from  the  population  is 
U*!V,  and  the  proportion  due  to  the  latent  nature  of  0 is  (1+M'')5/V.  The  results  are  shown  in  Table  1 1-1. 
The  value  of  U*IV  roughly  corresponds  to  reliability  in  classical  test  theory  and  indicates  the  amount  of 
information  about  an  average  individual’s  0 present  in  the  observed  responses  of  the  individual.  It  should 
be  recalled  again  that  the  objective  of  NAEP  is  not  to  estimate  and  compare  values  of  individual 
examinees,  the  accuracy  of  which  is  gauged  by  reliability  coefficients.  The  objective  of  NAEP,  rather,  is 
to  estimate  population  and  subpopulation  characteristics,  and  the  marginal  estimation  methods  described 
above  have  been  designed  to  do  so  consistently  regardless  of  the  values  of  reliability  coefficients. 


Table  11-1 

Estimation  Error  Variance  and  Related  Coefficients  for  the  1992  Grade  4 Reading  Composite 

(Based  on  Five  Plausible  Values) 


Proportion  of  Variance  Due  to... 
Student 

u* 

(l+5-')R 

V 

Sampling: 

U*IV 

Latency  of  0: 
(l+5-')B/V 

0.833 

0.076 

0.908 

0.92 

0.08 

Chapters  12  through  17  and  Appendix  E provide  values  of  the  proportion  of  variance  due  to 
sampling  and  due  to  the  latent  nature  of  0 for  all  1996  scales  and  composites  for  the  populations  as  a 
whole  and,  in  the  appendix,  for  selected  subpopulations.  It  will  be  seen  that  the  proportion  of  variance 
due  to  the  latency  of  0 varies  somewhat  among  subject  areas,  tending  to  be  largest  for  the  long-term  trend 
writing  assessment,  where  there  is  low  correlation  between  tasks  and  each  student  responded  to  only  one 
or  at  most  two  tasks.  The  proportion  of  variance  due  to  latency  of  0 is  smallest  for  the  composites  of  the 
main  assessment  subjects,  where  the  number  of  items  per  student  is  largest.  Essentially,  the  variance  due 
to  the  latent  nature  of  0 is  largest  when  there  is  less  information  about  a student’s  proficiency.  (Note  the 
distinction  between  estimation  error  variance  of  a parameter  estimate  and  the  estimate  of  the  variance  of 
the  0 distribution.  The  former  depends  on  the  accuracy  of  measurement;  the  large-sample  model-based 
expected  value  of  the  latter  does  not.)  Given  fixed  assessment  time,  this  decrease  in  information  will 
occur  whenever  the  amount  of  information  per  unit  time  decreases  as  can  happen  when  many  short 
constructed-response  or  multiple-choice  items  are  replaced  by  a few  extended  constructed-response 
items. 


11.5  DESCRIBING  STUDENT  PERFORMANCE 

Since  its  beginning,  a goal  of  NAEP  has  been  to  inform  the  public  about  what  students  in 
American  schools  know  and  can  do.  While  the  NAEP  scales  provide  information  about  the  distributions 
of  proficiency  for  the  various  subpopulations,  they  do  not  directly  provide  information  about  the  meaning 
of  various  points  on  the  scale.  Traditionally,  meaning  has  been  attached  to  educational  scales  by  norm- 
referencing— that  is,  by  comparing  students  at  a particular  scale  level  to  other  students.  In  contrast, 

NAEP  achievement  levels  and  scale  anchors  describe  selected  points  on  the  scale  in  terms  of  the  types  of 
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skills  that  are  likely  to  be  exhibited  by  students  scoring  at  that  level.  The  achievement  level  process  was 
applied  to  the  science  composite.  Scale  anchoring  of  certain  percentiles  of  the  student  proficiency 
distribution  was  applied  to  the  long-term  trend  assessment  composite  using  the  1996  data.  The 
achievement  level  process  for  mathematics  was  completed  for  the  1990  assessment  when  the  NAEP 
mathematics  framework  was  revised,  so  the  results  were  directly  applied  to  the  1996  results.  In  addition, 
each  item  was  mapped  to  a point  on  the  scale  in  which  it  belonged,  so  that  the  content  of  each  item 
provides  information  about  what  students  at  each  score  level  can  do  in  a probabistic  sense. 


11.5.1  Achievement  Levels 

NAGB  has  determined  that  achievement  levels  shall  be  the  first  and  primary  way  of  reporting 
NAEP  results.  Setting  achievement  levels  is  a method  for  setting  standards  on  the  NAEP  assessment  that 
identify  what  students  should  know  and  be  able  to  do  at  various  points  on  the  composite.  For  each  grade 
of  each  subject,  three  levels  were  defined — basic,  proficient,  and  advanced.  Based  on  initial  policy 
definitions  of  these  levels,  panelists  were  asked  to  determine  operational  descriptions  of  the  levels 
appropriate  with  the  content  and  skills  assessed  in  the  assessment.  With  these  descriptions  in  mind,  the 
panelists  were  then  asked  to  rate  the  assessment  items  in  terms  of  the  expected  performance  of 
marginally  acceptable  examinees  at  each  of  these  three  levels.  These  ratings  were  then  mapped  onto  the 
NAEP  scale  to  obtain  the  achievement  level  cutpoints  for  reporting.  Further  details  of  the  achievement 
level-setting  process  for  science  appear  in  Appendix  G. 


11.5.2  Performance  Descriptions  Based  on  Composite  Scales 

A procedure  known  as  scale  anchoring  was  used  to  develop  descriptions  of  student  performance 
at  selected  points  on  the  composite  scales.  The  scale  points  that  were  selected  for  anchoring  reflect  three 
levels  of  knowledge  and  abilities  corresponding  to  lower-,  middle-,  and  higher-performing  students  for 
each  subject. 

Around  each  percentile  point,  a band  was  built  to  define  a range  of  scale  scores.  Students 
described  as  being  at  a particular  level  were  within  a five  percentile  point  range  on  either  side  of  the 
specified  scale  point.  For  example,  the  50th  percentile  was  defined  as  the  region  between  the  45th  and 
55th  percentile  points  on  the  scale.  A question  was  identified  as  anchoring  at  a percentile  point  on  the 
scale  if  it  was  answered  successfully  by  at  least  65  percent  of  the  students  within  that  percentile  band. 
(The  criterion  was  set  at  74  percent  for  multiple-choice  questions  to  correct  for  the  possibility  of 
answering  correctly  by  guessing.) 

After  defining  the  bands  of  the  scale  to  be  anchored,  the  next  step  in  the  process  was  to  identify  : 
(1)  questions  answered  correctly  for  dichotomously  scored  questions,  or  (2)  questions  answered  at  a 
particular  score  level  for  partial  credit  constructed-response  questions.  Because  the  extended 
constructed-response  questions  were  scored  according  to  four  levels  of  performance,  each  extended 
constructed-response  question  was  treated  as  three  distinct  questions  corresponding  to  scores  of  Partial  or 
better.  Essential  or  better,  and  Extensive.  These  distinct  score  levels  were  then  analyzed  in  the  same 
manner  as  questions  scored  dichotomously,  as  either  correct  or  incorrect.  Thus,  for  example,  an  extended 
constructed-response  question  might  anchor  at  the  50th  percentile  for  Partial  or  better  responses  and  at 
the  90th  percentile  for  Essential  or  better  responses. 

A committee  of  subject  area  experts,  including  teachers  for  the  grades  involved,  college 
professors,  state  curriculum  supervisors,  and  researchers,  was  assembled  to  review  the  sets  of  questions 
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identified  for  each  percentile  band.  The  committee  was  divided  into  three  groups,  one  for  each  grade. 
Each  group  examined  and  analyzed  questions  that  anchored  at  the  25th,  50th,  and  90th  percentiles  to 
determine  the  specific  knowledge  and  abilities  associated  with  each  question. 

Committee  members  were  also  provided  with  the  sets  of  questions  at  each  grade  that  “did  not 
anchor  to  inform  their  decisions  about  what  students  could  do  by  seeing  examples  of  what  they  could 
not  do.  Drawing  on  their  knowledge  of  the  subject  area,  committee  members  were  asked  to  summarize 
student  performance  by  describing  the  knowledge,  skills,  and  abilities  demonstrated  by  students  in  each 
of  the  score  bands. 

The  performance  descriptions  are  cumulative;  that  is,  the  abilities  described  for  the  lower 
performing  students  are  considered  to  be  among  the  abilities  of  students  performing  at  higher  points  on 
the  scale.  Therefore,  the  full  description  of  student’s  knowledge  and  abilities  in  the  middle  scale  band 
would  include  those  abilities  described  at  the  lower  band.  Similarly,  the  abilities  of  students  performing 
at  the  higher  scale  band  include  the  abilities  described  for  students  at  the  middle  and  lower  bands. 


11.5.3  Item  Mapping  Procedures 

In  order  to  map  items  (questions)  to  particular  points  on  each  subject  area  scale,  a response 
probability  convention  had  to  be  adopted  that  would  divide  those  who  had  a higher  probability  of  success 
from  those  who  had  a lower  probability.  Establishing  a response  probability  convention  has  an  impact  on 
the  mapping  of  assessment  items  onto  the  scales.  A lower  boundary  convention  maps  the  items  at  lower 
points  along  the  scales,  and  a higher  boundary  convention  maps  the  same  items  at  higher  points  along  the 
scales.  The  underlying  distribution  of  skills  in  the  population  does  not  change,  but  the  choice  of  a 
response  probability  convention  does  have  an  impact  on  the  proportion  of  the  student  population  that  is 
reported  as  “able  to  do”  the  items  on  the  scales. 

There  is  no  obvious  choice  of  a point  along  the  probability  scale  that  is  clearly  superior  to  any 
other  point.  If  the  convention  were  set  with  a boundary  at  50  percent,  those  above  the  boundary  would  be 
more  likely  to  get  an  item  right  than  get  it  wrong,  while  those  below  that  boundary  would  be  more  likely 
to  get  the  item  wrong  than  right.  While  this  convention  has  some  intuitive  appeal,  it  was  rejected  on  the 
grounds  that  having  a 50/50  chance  of  getting  the  item  right  shows  an  insufficient  degree  of  mastery.  If 
the  convention  were  set  with  a boundary  at  80  percent,  students  above  the  criterion  would  have  a high 
probability  of  success  with  an  item.  However,  many  of  the  students  below  this  criterion  show  some  level 
of  achievement  that  would  be  ignored  by  such  a stringent  criterion.  In  particular,  those  in  the  range 
between  50  and  80  percent  correct  would  be  more  likely  to  get  the  item  right  than  wrong,  yet  would  not 
be  in  the  group  described  as  “able  to  do”  the  item. 

In  a compromise  between  the  50  percent  and  the  80  percent  conventions,  NAEP  has  adopted  two 
related  response  probability  conventions:  74  percent  for  multiple-choice  items  (to  correct  for  the 
possibility  of  answering  correctly  by  guessing),  and  65  percent  for  constructed-response  items  (where 
guessing  is  not  a factor).  These  probability  conventions  were  established,  in  part,  based  on  an  intuitive 
judgment  that  they  would  provide  the  best  picture  of  students’  knowledge  and  skills. 

Some  additional  support  for  the  dual  conventions  adopted  by  NAEP  was  provided  by  Huynh 
(1998,  1994).  He  examined  the  IRT  information  provided  by  items,  according  to  the  IRT  model  used  in 
scaling  NAEP  items.  Following  Bock  (1972),  Huynh  decomposed  the  item  information  into  that  provided 
by  a correct  response  [Py,  (6)  *Ij  (0)]  and  that  provided  by  an  incorrect  response  [(1-P  (d))  *I  (0)].  Huynh 
showed  that  the  item  information  provided  by  a correct  response  to  a constructed-response  item  is 
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maximized  at  the  point  along  the  scale  at  which  two-thirds  of  the  students  get  the  item  correct  (for 
multiple-choice  items  with  four  options,  information  is  maximized  at  the  point  at  which  75  percent  get 
the  item  correct).  Maximizing  the  item  information,  I (6),  rather  than  the  information  provided  by  a 
correct  response  [P  (0)  * I (6)],  would  imply  an  item  mapping  criterion  closer  to  50  percent.  Maximizing 
the  item  information,  I (6),  takes  into  account  both  responses  that  are  correct  and  those  that  are  incorrect, 
however. 


For  dichotomously  scored  items  the  information  function  as  defined  by  Bimbaum  (1968,  p.  463) 
is  defined  for  the  yth  item  as 


m = 


(1.7a,)V;o(ft)[P„(a)-cJ 


where  the  notation  is  the  same  as  that  used  in  Equations  (11.1)  and  (1 1 .2).  The  item  information  function 
was  defined  by  Samejima  (1969)  in  general  for  polytomously  scored  items,  and  has  been  derived  for 
items  scaled  by  the  generalized  partial  credit  model  (Muraki,  1993;  Donoghue,  1994)  as  (in  a slightly 
different,  but  equivalent  form) 


<■=0  I i=o  J 


11.6  OVERVIEW  OF  THE  1996  NAEP  SCALES 

The  following  IRT  scale-score  analyses  were  carried  out  for  the  1996  NAEP  assessment: 

• Mathematics:  Five  IRT  scales  linked  back  to  the  1990  and  1992  main  assessment  of 
mathematics  and  one  unidimensional  IRT  mathematics  scale  linking  1996  results  to 
results  from  mathematics  assessments  in  1973,  1976,  1982,  1986,  1990,  1992,  and 
1994.  The  first  five  scales,  along  with  a composite  scale,  are  associated  with  the 
1996  main  assessment,  while  the  unidimensional  scale  is  associated  with  the  1996 
long-term  trend  assessment. 

• Science:  Three  newly  developed  IRT  scales  for  the  main  assessment  of  science  and 
one  unidimensional  scale  linking  1996  results  to  results  from  science  assessments  in 
1969,  1973,  1977,  1982,  1986,  1990,  1992,  and  1994.  The  first  three  scales,  along 
with  a composite  scale,  are  associated  with  the  1996  main  assessment,  while  the 
unidimensional  scale  is  associated  with  the  1996  long-term  trend  assessment. 

• Long-Term  Trend  Reading:  One  IRT  scale  linking  1996  results  to  results  from 
reading  assessments  in  1971,  1975,  1979,  1984,  1988,  1990,  1992,  and  1994.  This 
scale  is  associated  with  the  1996  long-term  trend  assessment. 

• Long-Term  Trend  Writing:  One  polytomous  item  scale  linking  1996  writing  results 
to  the  1984,  1988,  1990,  1992,  and  1994  assessments.  This  scale  is  associated  with 
the  1996  long-term  trend  assessment. 


Details  follow  in  Chapters  12  through  17. 
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Chapter  12 

DATA  ANALYSIS  FOR  THE  MATHEMATICS  ASSESSMENT* 

Frank  Jenkins,  Hua-Hua  Chang,  and  Edward  Kulick 
Educational  Testing  Service 


12.1  INTRODUCTION 

This  chapter  describes  the  analyses  performed  on  the  responses  to  the  cognitive  and  background 
items  in  the  1996  mathematics  assessment.  This  chapter  focuses  on  the  methods  and  procedures  used  to 
estimate  IRT-based  scale  score  distributions  for  subgroups  of  students.  This  includes  a wide  array  of 
topics,  such  as  the  scoring  of  constructed-response  items,  classical  item  characteristics,  item 
responsetheory  (IRT)  analysis  of  mathematics  scales,  and  estimation  of  subgroup  means  by  the 
imputation  of  plausible  values.  The  statistical  bases  of  the  IRT  and  plausible  values  methodology 
described  in  this  chapter  are  given  in  Chapter  11.  These  analyses  led  to  the  results  presented  in  NAEP 
1996  Mathematics  Report  Card  for  the  Nation  and  the  States  (Reese,  Miller,  Mazzeo,  & Dossey,  1997). 
For  a description  of  the  state  analyses,  see  the  Technical  Report  of  the  NAEP  1996  State  Assessment 
Program  in  Mathematics  (Allen,  Jenkins,  Kulick,  & Zelenak,  1997). 

The  student  samples  that  were  administered  mathematics  items  in  the  1996  national  assessment 
are  shown  in  Table  12-1.  (See  Chapters  1 and  3 for  descriptions  of  the  target  populations  and  the  sample 
design  used  for  the  assessment.).  These  samples  were  defined  only  by  grade  (4,  8,  or  12)  and  not  by  age 
of  the  student.  Data  from  the  Math  Main  samples  comprised  the  spiraled  balanced  incomplete  block 
(BIB)  design.  The  present  chapter  contains  information  about  the  scaling  of  data  from  these  samples.  The 
long-term  trend  analyses  are  presented  in  a separate  chapter  (see  Chapter  15).  The  other  samples  (Math- 
Estimation,  Math-Theme,  and  Math-Advanced)  will  be  analyzed  and  presented  in  separate  focus  reports. 
A brief  description  of  the  analyses  of  these  samples  is  presented  in  Section  12.3.  Technical 
documentation  detailing  the  analyses  of  the  1996  State  Assessment  of  mathematics  is  provided  in  the 
Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Mathematics  (Allen,  Jenkins,  Kulick, 
& Zelenak,  1997). 


' Frank  Jenkins  was  the  primary  person  responsible  for  the  planning,  specifications,  and  coordination  of  the  national  mathematics 
analyses.  He  was  assisted  by  Hua-Hua  Chang.  Data  analysis  and  scaling  were  coordinated  by  Ed  Kulick  with  help  from  Steve  Wang 
and  Xiaohui  Wang  and  additional  assistance  from  David  Freund. 


255 


ERIC 


280 


Table  12-1 

NAEP  1996  Mathematics  Student  Samples 


Sample 

Booklet 

Number 

Mode 

Cohort 

Assessed 

Time  of  Testing’ 

Number 

Assessed 

4 [Math  Main] 

1-26 

Print 

Grade  4 

1/3/96  - 3/29/96  (Winter) 

6,627 

8 [Math  Main] 

1-26 

Print 

Grade  8 

1/3/96  - 3/29/96  (Winter) 

7,146 

12  [Math  Main] 

1-26 

Print 

Grade  12 

1/3/96  - 3/29/96  (Winter) 

6,904 

4 [Math-Estimation] 

127 

Tape 

Grade  4 

1/3/96-3/29/96  (Winter) 

2,023 

8 [Math-Estimation] 

127 

Tape 

Grade  8 

1/3/96  - 3/29/96  (Winter) 

2,183 

12  [Math-Estimation] 

127 

Tape 

Grade  12 

1/3/96  - 3/29/96  (Winter) 

1,849 

4 [Math-Theme] 

128,129 

Print 

Grade  4 

1/3/96  - 3/29/96  (Winter) 

3,790 

8 [Math-Theme] 

128,129 

Print 

Grade  8. 

1/3/96  - 3/29/96  (Winter) 

4,027 

12  [Math-Theme] 

128,129 

Print 

Grade  12 

1/3/96  - 3/29/96  (Winter) 

3,735 

8 [Math-Advanced] 

130 

Print 

Grade  8 

1/3/96  - 3/29/96  (Winter) 

2,337 

12  [Math-Advanced] 

130 

Print 

Grade  12 

1/3/96  - 3/29/96  (Winter) 

2,965 

‘ Final  makeup  sessions  were  held  April  1-5,  1996. 


LEGEND: 


Math 

Main 

Estimation 

Advanced 


Mathematics 

Main  spiral  BIB  assessment 
Main  estimation  assessment 
Assessment  with  advanced  booklets 


Print  Printed  administration 

Tape  Audiotape  administration 

Theme  Assessment  with  theme  booklets 


12.2  DESCRIPTION  OF  STUDENT  SAMPLE,  ITEMS,  AND  ASSESSMENT 
BOOKLETS 

The  data  from  the  main  BIB  mathematics  assessment  (from  samples  4[Math  Main],  8 [Math 
Main],  and  12[Math  Main])  were  used  for  main  analyses  comparing  the  levels  of  mathematics 
achievement  for  various  subgroups  of  the  1996  target  populations.  In  previous  assessments  the 
mathematics  samples  were  defined  as  age/grade  cohorts  (e.g.,  students  who  were  either  in  the  fourth 
grade  or  9 years  old).  Starting  with  the  1996  assessment,  cohorts  were  defined  solely  by  grade.  The 
sampled  students  in  each  of  these  three  grade  cohorts  were  assessed  in  the  winter.  The  samples  in  the 
main  assessment  are  presented  in  Table  12-1. 

The  pool  of  items  used  in  the  1996  mathematics  assessment  contained  a range  of  constructed- 
response  and  multiple-choice  questions  measuring  performance  on  sets  of  objectives  (National 
Assessment  Governing  Board,  1994).  The  framework  for  the  objectives  is  described  in  Chapter  2.  A total 
of  360  distinct  mathematics  items  addressing  these  objectives  were  scaled  (see  Table  12-2).  The  number 
of  items  per  grade  was  144,  164,  and  165  respectively  for  grades  4,  8,  and  12  (before  scaling).  Tables  12- 
3,  12-5,  and  12-7  give,  for  each  grade,  the  numbers  of  items  by  item  type  and  block  (before  scaling).  For 
item  counts  after  scaling,  taking  into  account  items  that  were  dropped  or  collapsed,  see  Tables  12-4,  12- 
6,  and  12-8.  These  tables  indicate  that  there  was  a nearly  fourfold  increase  in  the  number  of 
polytomously-scored  constructed-response  items  over  the  numbers  presented  in  the  1992  assessment.  In 
1996,  the  number  of  items  was  30  in  grade  4,  30  in  grade  8,  and  33  in  grade  12.  The  items  were  classified 
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into  five  content  strands:  numbers  and  operations;  measurement;  geometry;  data  analysis,  statistics,  and 
probability;  and  algebra  and  functions.  These  five  content  strands  (renamed  number  sense,  properties, 
and  operations;  measurement;  geometry  and  spatial  sense;  data  analysis,  statistics,  and  probability;  and 
algebra' and  functions  in  the  1996  analysis)  constituted  the  scales  used  in  1996  reporting.  The  items 
were  partitioned  into  13  blocks  and  these  blocks  were  then  assigned  to  26  booklets  according  to  a BIB 
design.  Each  booklet  contained  relatively  few  items  from  each  of  the  five  categories. 


Table  12-2 

National  Main  BIB:  Numbers  of  Scaled  Mathematics  Items 
Common  Across  Grade  Levels,  by  Content  Strand  Scale 


Grade(s) 

Number  Sense, 
Properties, 
and 

Operations 

Measurement 

Geometry 

and 

Spatial 

Sense 

Data  Analysis, 
Statistics, 
and 

Probability 

Algebra 

and 

Functions 

Total 

4 only 

40 

20 

12 

11 

8 

91 

8 only 

20 

13 

12 

10 

18 

73 

12  only 

23 

11 

17 

22 

32 

105 

4 and  8 

10 

2 

9 

3 

7 

31 

8 and  12 

8 

9 

6 

9 

6 

38 

4 and  12 

0 

0 

0 

0 

0 

0 

4,  8,  and  12 

9 

3 

4 

3 

3 

22 

Total  Grade  4 

59 

25 

25 

17 

18 

144 

Total  Grade  8 

47 

27 

31 

25 

34 

164 

Total  Grade  12 

40 

23 

27 

34 

41 

165 

Total 

110 

58 

60 

58 

74 

360 

In  the  main  samples,  each  student  was  administered  a booklet  containing  three  blocks  of 
mathematics  cognitive  items,  a block  of  background  questions  common  to  all  booklets  for  a particular 
grade  level,  and  a block  of  mathematics-related  background  questions  common  to  all  mathematics 
booklets  for  a particular  grade  level.  At  the  end  of  each  booklet  there  was  a short  block  consisting  of  five 
questions  concerning  the  student’s  motivation  and  his  or  her  perception  of  the  difficulty  of  the  cognitive 
items.  The  BIB  design  by  which  the  13  blocks  of  mathematics  cognitive  items  were  assigned  to  the  26 
booklets  for  each  grade  level  is  discussed  in  Chapter  4.  The  13  blocks  were  not  intended  to  be  parallel 
measuring  instruments.  For  example,  several  blocks  contained  only  the  items  designed  for  calculator 
usage,  and  some  blocks  contained  items  for  ruler  and  protractor  usage.  In  addition,  the  proportion  of 
items  sampled  from  the  five  categories  were  not  exactly  the  same  among  the  13  blocks. 

The  360  unique  items  were  constructed  according  to  several  formats,  some  traditional  and  some 
more  innovative:  multiple-choice  items,  constfucted-response  items  scored  dichotomously,  constructed- 
response  items  scored  polytomously,  and  cluster  items.^  The  multiple-choice  items  conformed  to  the 
familiar  format  of  a stem  followed  by  several  possible  answers,  with  only  one  answer  being  correct. 


^ The  content  strand  number  sense,  properties,  and  operations  was  called  numbers  and  operations  in  the  1990  and  1992 
assessments.  The  content  strand  geometry  and  spatial  sense  was  called  geometry  in  the  1990  and  1992  assessments. 

^ A cluster  item  is  an  aggregation  of  a group  of  items  (in  the  case  of  NAEP  mathematics,  typically  three  to  five  items)  that  are 
related  to  a single  content  strand,  topic,  or  stimulus,  and  are  developed  and  scored  as  a single  unit  (see  Wainer  & Kiely,  1987,  for 
further  details  and  examples  of  different  types  of  cluster  items). 


257 


Constructed-response  items  that  were  scored  dichotomously  were  questions  that  required  an  open-ended 
response  (e.g.,  explaining  why  the  previous  question  was  answered  as  it  was).  These  questions  were  read 
by  raters  who  determined  whether  or  not  the  response  was  correct.  Constructed-response  items  that  were 
scored  polytomously  were  open-ended  questions  that  required  several  stages  of  reasoning  or  problem 
solving.  They  were  also-read  by  raters  but  were  given  a score  reflecting  degree  of  correctness  rather  than 
simply  judged  right  or  wrong.  Cluster  items  were  derived  from  a set  of  three  to  five  multiple-choice  items 
that  related  to  the  same  basic  stem.  The  cluster  score  was  the  number  of  constituent  cluster  items  that  the 
respondent  answered  correctly. 

As  Table  12-3  indicates,  of  the  144  items  at  grade  4,  79  were  multiple-choice  items,  16  were 
constructed  response  items  that  were  scored  dichotomously,  48  were  constructed-response  items  that 
were  scored  polytomously,  and  1 was  a cluster  item.  As  shown  in  Table  12-5,  of  163  items  at  grade  8,  91 
were  multiple-choice  items,  21  were  constructed-response  items  that  were  scored  dichotomously,  48 
were  constructed-response  items  that  were  scored  polytomously,  and  3 were  cluster  items.  For  grade  12, 
Table  12-7  indicates  that  of  the  166  items  administered,  92  were  multiple-choice  items,  21  were 
constructed-response  items  that  were  scored  dichotomously,  50  were  constructed-response  items  that 
were  scored  polytomously,  and  3 were  cluster  items. 

Tables  12-4,  12-6,  and  12-8  show  comparable  information  for  each  grade  after  the  scaling  was 
carried  out. 


Table  12-3 

7996  NAEP  Mathematics  Block  Composition  by  Content  Strand 
and  Item  Type,  Grade  4,  As  Defined  Before  Scaling 


Multiple- 

Choice 

Constructed-Response  Items  Scored 
Polytomously 

Cluster 

Total 

Block 

Items 

2-category  3-category  4-category  5-category 

Items 

Items 

M3 

9 

4 

0 

0 

0 

0 

13 

M4 

14 

0 

0 

0 

0 

0 

14 

M5 

4 

0 

5 

0 

1 

0 

10 

M6 

0 

7 

2 

2 

0 . 

0 

11 

M7 

3 

0 

4 

0 

1 

0 

8 

M8 

14 

0 

1 

0 

0 

0 

15 

M9 

6 

2 

0 

0 

1 

1 

10 

MIO 

0 

1 

4 

1 

0 

0 

6 

Mil 

11 

0 

0 

2 

3 

0 

16 

M12 

. 5 

0 

3 

0 

1 

0 

9 

M13 

6 

1 

2 

2 

1 

0 

12 

M14 

4 

0 

5 

0 

1 

0 

10 

M15 

3 

1 

5 

0 

1 

0 

10 

Total 

79 

16 

31 

7 

10 

1 

144 
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Table  12-4 

1996  NAEP  Mathematics  Block  Composition  by  Content  Strand 
and  Item  Type,  Grade  4,  As  Defined  After  Scaling* 


Multiple- 

Choice 

Constructed-Response  Items  Scored 
Polytomously 

Cluster 

Total 

Block 

Items 

2-category 

3-category  4-^ 

category  1 

5-category 

Items 

Items 

M3 

9 

4 

, 0 

0 

0 

0 

13 

M4 

14 

0 

0 

0 

0 

0 

14 

M5 

4 

0 

5 

0 

1 

0. 

10 

M6 

0 

7 

2 

2 

0 

0 

11 

M7 

3 

2 

2 

0 

1 

0 

8 

M8 

14 

0 

1 

0 

0 

0 

15 

M9 

6 

2 

0 

1 

0 

1 

10 

MIO 

0 

1 

4 

1 

0 

0 

6 

Mil 

11 

0 

0 

2 

3 

0 

16 

M12 

5 

0 

3 

0 

1 

0 

9 

M13 

6 

1 

2 

2 

1 

0 

12 

M14 

4 

0 

5 

1 

0 

0 

10 

M15 

3 

2 

4 

0 

1 

0 

10 

Total 

79 

19 

28 

9 

8 

1 

144 

* Counts  reflect  items  that  were  dropped  and  collapsed. 

Table  12-5 

1996  NAEP  Mathematics  Block  Composition  by  Content  Strand 
and  Item  Type,  Grade,  As  Defined  Before  Scaling 

Multiple- 

Choice 

Constructed-Response  Items  Scored 
Polytomously 

Cluster 

Total 

Block 

Items 

2-category  3-category  4-category  5-category 

Items 

Items 

M3 

8 

2 

0 

1 

1 

1 

13 

M4 

21 

0 

0 

0 

0 

0 

21 

M5 

6 

0 

4 

0 

1 

0 

11 

M6 

0 

11 

4 

1 

0 

0 

16 

M7 

4 

0 

5 

0 

1 

0 

10 

M8 

16 

1 

0 

1 

0 

0 

18 

M9 

5 

3 

0 

0 

1 

0 

9 

MIO 

0 

1 

5 

1 

0 

0 

7 

Mil 

13 

0 

1 

2 

3 

0 

19 

M12 

4 

0 

4 

1 

0 

0 

9 

M13 

6 

3 

1 

0 

1 

0 

11 

M14 

5 

0 

3 

0 

1 

1 

10 

M15 

3 

0 

4 

0 

1 

1 

9 

Total 

91 

21 

31 

7 

10 

3 

163 
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Table  12-6 

1996  NAEP  Mathematics  Block  Composition  by  Content  Strand 
and  Item  Type,  Grade  8,  Defined  Afier  Scaling* 


Multiple- 

Choice 

Constructed-Response  Items  Scored 
Polytomously 

Cluster 

Total 

Block 

Items 

2-category 

3-category  4' 

■category 

5-category 

Items 

Items 

M3 

8 

2 

0 

2 

0 

1 

13 

M4 

21 

0 

0 

0 

0 

0 

21 

M5 

6 

1 

4 

0 

0 

0 

11 

M6 

0 

11 

4 

1 

0 

0 

16 

M7 

4 

1 

5 

0 

0 

0 

10 

M8 

16 

1 

0 

1 

0 

0 

18 

M9 

5 

3 

0 

1 

0 

0 

9 

MIO 

0 

1 

5 

1 

0 

0 

7 

Mil 

13 

0 

1 

2 

3 

0 

19 

M12 

4 

2 

3 

0 

0 

0 

9 

M13 

6 

3 

1 

1 

0 

0 

11 

M14 

5 

1 

1 

1 

0 

1 

9 

M15 

3 

0 

4 

1 

0 

1 

9 

Total 

91 

26 

28 

11 

3 

3 

162 

* Counts  reflect  items  that  were  dropped  and  collapsed. 

Table  12-7 

7996  NAEP  Mathematics  Block  Composition  by  Content  Strand 
and  Item  Type,  Grade  12,  Defined  Before  Scaling 

Multiple 

Choice 

Constructed-Response  Items  Scored 
Polytomously 

Cluster 

Total 

Block 

Items 

2-category  3-category  4-category  5-category 

Items 

Items 

M3 

10 

3 

1 

0 

0 

0 

14 

M4 

22 

0 

0 

0 

0 

0 

22 

M5 

4 

0 

5 

0 

1 

0 

10 

M6 

0 

13 

2 

2 

0 

0 

17 

M7 

3 

0 

4 

0 

1 

1 

9 

M8 

17 

2 

1 

1 

0 

0 

21 

M9 

6 

2 

0 

1 

0 

0 

9 

MIO 

3 

0 

5 

1 

1 

0 

10 

Mil 

11 

1 

2 

0 

0 

0 

14 

M12 

4 

0 

4 

0 

1 

1 

10 

M13 

3 

0 

0 

3 

3 

0 

9 

M14 

5 

0 

4 

0 

1 

1 

11 

M15 

4 

0 

5 

0 

1 

0 

10 

Total 

92 

21 

33 

8 

9 

3 

166 

260 


Table  12-8 

1996  NAEP  Mathematics  Block  Composition  by  Content  Strand 
and  Item  Type,  Grade  12,  ^45  Defined  After  Scaling* 


Multiple- 

Choice 

Constructed-Response  Items  Scored 
Polytomously 

Cluster 

Total 

Block 

Items 

2-category 

3-category 

4-category  i 

5-category 

Items 

Items 

M3 

10 

3 

1 

0 

0 

0 

14 

M4 

22 

0 

0 

0 

0 

0 

22 

M5 

4 

0 

6 

0 

0 

0 

10 

M6 

0 

13 

2 

2 

0 

0 

17 

M7 

3 

0 

4 

1 

0 

1 

9 

M8 

17 

2 

1 

1 

0 

0 

21 

M9 

6 

2 

0 

1 

0 

0 

9 

MIO 

3 

0 

5 

2 

0 

0 

10 

Mil 

11 

1 

2 

0 

0 

0 

14 

M12 

4 

0 

4 

1 

0 

1 

10 

M13 

3 

0 

0 

4 

2 

0 

9 

M14 

4 

1 

3 

1 

0 

1 

10 

M15 

4 

1 

5 

0 

0 

0 

10 

Total 

91 

23 

33 

13 

2 

3 

165 

* Counts  reflect  items  that  were  dropped  and  collapsed. 


12.3  SPECIAL  MATHEMATICS  ASSESSMENTS 

There  were  three  special  studies  in  the  1996  NAEP  assessment — estimation,  theme,  and  advanced 
studies.  The  block  structure  of  the  special  study  booklets  is  provided  only  in  Table  12-9.  The  estimation  and 
the  two  theme  blocks  have  a linking  BIB  block,  M4,  in  the  first  position.  This  structure  held  for  all  three 
grades,  but  the  actual  items  in  the  blocks  varied  over  grades.  In  addition  to  the  cognitive  blocks,  each  of  the 
special  study  booklets  included  three  blocks  in  common  with  the  main  assessment: 

♦ a general  student  background  block  (BMl), 

♦ a mathematics  background  block  (MBl),  and 

♦ a motivation  block  (M2). 


Table  12-9 

Block  Structure  of  the  Special  Study  Booklets 


Study 

Booklet 

Cognitive  Blocks 

Estimation 

■ 127 

M4,  M16,  M17 

Theme 

128 

M4,  M21 

Theme 

129 

M4,  M22 

Advanced  Mathematics' 

130 

M20,  M18,  M19 

' Grades  8 and  1 2 only 
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The  cognitive  portion  of  the  estimation  booklet  was  administered  in  two  sections;  first  there  was  a regular 
mathematics  block  in  common  with  the  main  assessment  that  was  self-paced;  and  second,  there  were  two 
estimation  blocks  (M16  and  M17)  in  which  items  were  administered  by  a paced  audio  tape.  The  theme  and 
advanced  booklets  were  self-paced.  Note  that  the  theme  booklets  contained  a single  non-theme  block  (M4) 
in  common  with  the  main  assessment. 

The  special  studies  were  not  part  of  the  main  assessment;  analysis  for  these  booklets  will  appear 
in  separate  focus  reports.  The  major  findings  for  these  studies  will  be  derived  from  an  analysis  of  the 
‘reporting’  samples,  which  are  subsamples  of  the  assessment’s  total  sample.  The  nonreporting  segments 
of  the  total  samples  were  added  to  the  assessments  in  order  to  study  the  effects  of  changing  inclusion 
rules  and  accommodations  for  students  of  limited  English  proficiency  (LEP)  and  students  with 
disabilities  (SD)  (See  Section  12.4). 

The  estimation  and  theme  samples  were  drawn  from  the  same  population  as  the  main  assessment. 
The  grades  8 and  12  advanced  booklets  were  drawn  from  a population  of  students  considered  to  be  high 
mathematics  achievers.  Students  for  the  grade  8 study  of  advanced  mathematics  were  sampled  from  a 
population  of  students  who  were  enrolled  in  an  advanced  class  (defined  as  algebra  1 or  beyond)  during 
the  1995-96  school  year.  For  grade  12,  the  advanced  study  students  were  also  sampled  from  a population 
of  students  who  took  an  advanced  course  during  the  school  year.  Grade  12  advanced  courses  were; 
Algebra  3,  Pre-calculus,  Calculus  and  Analytical  Geometry,  Calculus,  and  AP  Calculus.  Table  12-10  lists 
the  number  of  items  in  the  special  assessment  blocks  in  the  three  grades.  More  detail  is  available  in  the 
procedural  appendices  of  reports  on  the  estimation,  theme  and  advanced  analyses. 


Table  12-10 

Number  of  Cognitive  Items  in  the 
1996  Special  Mathematics  Assessment  Blocks 

Estimation  Theme  Advanced 

Blocks  Blocks  Blocks 


M16 

M17 

M21 

M22 

M20 

M18 

M19 

Grade  4 

20 

14 

8 

6 

1 

1 

1 

Grade  8 

22 

16 

12 

15 

13 

10 

12 

Grade  12 

22 

16 

11 

7 

15 

11 

11 

* There  were  no  advanced  blocks  administered  to  grade  4 students. 


12.4  ASSESSING  THE  EFFECT  OF  CHANGING  INCLUSION  CRITERION  AND 
ACCOMMODATIONS  FOR  SD/LEP  STUDENTS 

NAEP  samples  include  SD/LEP  students  in  at  least  the  same  percentages  in  which  they  are  found 
in  the  general  school  population.  Although  a substantial  percentage  of  these  students  are  included  in  the 
assessment,  schools  are  allowed  to  exclude  some  of  these  students  from  NAEP  when  the  students  are 
judged  to  be  incapable  of  meaningfully  participating  in  a large  scale  assessment.  To  facilitate  the 
consistent  implementation  of  NAEP’s  intention  to  include  as  many  students  as  possible,  NAEP  provides 
specific  criteria  that  staff  in  sampled  schools  can  use  to  determine  who  should  be  included  in  the 
assessment. 
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In  the  1996  assessment,  procedures  for  dealing  with  SD  and  LEP  students  were  modified,  based 
on  recommendations  from  the  U.S.  Department  of  Education.  First,  inclusion  criteria  were  revised  with 
the  intention  of  making  them  clearer,  more  inclusive,  and  more  likely  to  be  applied  consistently  across 
schools.  Second,  a variety  of  assessment  accommodations  and  adaptations  were  offered  to 

♦ students  with  disabilities  whose  individualized  education  programs  (lEPs) 
specified  such  accommodations,  and 

♦ LEP  students  who  were  unable  to  take  the  assessment  in  English. 

In  order  to  assess  the  impact  of  new  inclusion  criteria  and  accommodations,  a three-sample 
design  was  instituted  at  all  three  grades.  The  first  sample  comprised  students  who  were  subject  to  the 
1994  inclusion  rules  and,  as  was  the  case  in  past  NAEP  assessments,  were  offered  no  accommodations. 
Students  in  the  second  sample  were  subject  to  the  1996  inclusion  rules  but  the  SD/LEP  students  were 
offered  no  accommodations.  The  third  sample  had  students  who  were  comprised  students  who  were 
assessed  under  conditions  that  will  probably  be  used  for  future  NAEP  assessments — new  inclusion  rules 
and  some  accommodations  being  offered  to  the  included  SD/LEP  students.  See  Chapter  3,  Sections  3.4  to 
3.8,  for  more  details. 

Results  of  this  study  can  be  found  in  the  focus  report  dealing  with  the  comparison  of  the  three 
accommodation  samples. 


12.5  ITEM  ANALYSIS 

This  section  contains  a detailed  description  of  the  item  analysis  performed  using  the  national 
main  BIB  sample  data.  The  analysis  examines  items  within  blocks.  In  preparation  for  this  analysis, 
constructed-response  items  with  more  than  two  categories  and  cluster  items  were  polytomously  scored, 
two-category  constructed-response  items  were  dichotomously  scored,  and  derived  background  variables 
were  calculated.  Item  statistics  such  as  mean  percent  correct,  average  score,  item  to  total  score 
correlations  and  percent  responding  in  each  item  category  were  calculated. 

Tables  12-11,  12-12,  and  12-13  show  the  number  of  items,  mean  proportion  correct,  mean  item  to 
total  score  correlation,  and  alpha  reliability  for  each  block  administered  at  each  grade  level  for  the  main 
assessment.  These  values  were  calculated  within  block  only  for  those  items  used  in  the  scaling  process. 
The  table  also  gives  the  number  of  students  who  were  administered  the  block  and  the  percent  not 
reaching  the  last  item  in  the  block.  Student  weights  were  used,  except  for  reporting  the  sample  sizes.  The 
results  for  the  blocks  administered  at  each  grade  level  indicated  that  the  blocks  differ  in  number  of  items, 
average  difficulty,  reliability,  and  percent  not  reaching  the  last  item. 

Tables  12-11  through  12-13  show  the  item  analyses  by  block  position  within  a booklet.  Each 
booklet  had  three  cognitive  blocks,  and  each  block  appeared  in  three  different  booklets — once  in  the  first, 
once  in  the  second,  and  once  in  the  third  position.  In  some  past  assessments  (e.g.,  1992  science),  students 
responded  differently  to  the  items  depending  on  the  block  position.  The  LA  tables  evidence  few 
systematic  differences  in  item  summary  statistics  as  a function  of  block  position.  For  grade  4 (Table  12- 
1 1)  the  weighted  average  item  score  had  a slight  tendency  to  be  higher  when  a block  was  in  the  first 
rather  than  the  third  position.  For  grades  8 and  12  no  systematic  difference  can  be  seen. 
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Table  12-11 

Descriptive  Statistics  for  Item  Blocks  by  Position  Within  Test  Booklet  and  Overall 
Occurrences  for  the  Mathematics  Main  Sample,  Grade  4,  As  Defined  After  Scaling 

Block  ’ Block 
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Table  12-12 

Descriptive  Statistics  for  Item  Blocks  by  Position  Within  Test  Booklet  and  Overall 
Occurrences  for  the  Mathematics  Main  Sample,  Grade  8,  As  Defined  After  Scaling 
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As  described  in  Chapter  9,  in  NAEP  analyses  (both  conventional  and  IRT-based),  a distinction  is 
made  between  missing  responses  at  the  end  of  each  block  (not-reached)  and  missing  responses  prior  to 
the  last  completed  response  (omitted).  Not-reached  items  are  those  occurring  after  the  last  item  the 
student  completed  in  a block.  Items  that  were  not  reached  are  treated  as  if  they  had  not  been  presented  to 
the  examinee,  while  omitted  items  are  regarded  as  incorrect.  The  proportion  of  students  attempting  the 
last  item  of  a block  (or,  equivalently,  1 minus  the  proportion  not  reaching  the  last  item)  is  often  used  as 
an  index  of  the  degree  of  speededness  of  the  block  of  items. 

Standard  practice  at  ETS  is  to  treat  all  students  who  did  not  respond  to  the  last  item  as  if  they  had 
not  reached  that  item.  For  multiple-choice  and  short  constructed-response  items,  this  convention 
produced  a reasonable  pattern  of  results,  in  that  the  proportion  of  students  reaching  the  last  item  does  not 
differ  markedly  from  the  proportion  attempting  the  next-to-last  item.  However,  for  the  blocks  that  ended 
with  extended  constructed-response  items,  this  convention  resulted  in  an  implausibly  large  drop  in  the 
number  of  students  attempting  the  final  item  (see  Koretz  et  al.,  1993).  Therefore,  for  blocks  that  ended 
with  an  extended  constructed-response  item,  students  who  attempted  the  next-to-last  item  but  did  not 
respond  to  the  last  item  were  classified  as  having  intentionally  omitted  that  item. 

Tables  12-1 1 to  12-13  also  contain  information  about  the  effect  of  the  position  of  blocks  within 
booklets  on  the  average  percent  correct  for  items  within  each  block  presented  to  the  BIB  samples  for 
each  grade.  The  averages  for  the  grade-only  portion  of  the  focused-BiB  samples  show  that  the  order  of 
blocks  within  booklets  did  not  have  a large  or  consistent  effect  on  scale  scores  in  the  mathematics 
focused-BIB  assessment. 


12.5.1  Constructed-Response  Items 

As  indicated  in  Tables  12-3  to  12-8,  over  40  percent  of  the  mathematics  items  were  constructed- 
response.  Constructed-response  items  that  were  scored  dichotomously  were  given  a right/wrong  scoring. 
The  categories  of  responses  for  the  items  and  the  number  of  responses  that  were  rescored  for  each  item 
are  indicated  in  Appendix  I.  The  percent  agreement  for  the  raters  and  Cohen’s  Kappa,  a reliability 
estimate  appropriate  for  items  that  are  dichotomized,  are  also  given  in  the  tables.  The  sample  sizes  listed 
in  the  tables  correspond  to  the  samples  used  in  calculating  the  rater  reliability. 

In  general,  the  rater  reliability  of  the  scoring  for  dichotomized  responses  was  quite  high.  Cohen’s 
Kappa  reliabilities  ranged  over  items  from  0.76  to  1.00  for  grade  4,  from  0.90  to  1.00  for  grade  8,  and 
from  0.67  to  1.00  for  grade  12. 

Chapter  7 discusses  the  definition  of  the  item  ratings  and  describes  the  process  by  which  teams 
of  raters  scored  the  constructed-response  items.  This  discussion  includes  the  rating  definitions  for  short 
and  extended  constructed-response  items  as  well  as  the  range  of  interrater  reliabilities  that  occurred. 
Extended  constructed-response  items  were  scored  on  a scale  from  1 to  5 to  reflect  degrees  of  knowledge. 
In  scaling,  this  scale  is  shifted  to  0 to  4.  Rating  information  on  extended  constructed-response  items  can 
be  found  in  Appendix  I,  which  lists  the  sample  sizes,  percent  agreement,  and  Cohen’s  Kappa  reliability 
index. 
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12.5.2  Differential  Item  Function  Analysis 


A differential  item  functioning  (DIF)  analysis  of  the  main-assessment  mathematics  items  was 
done  in  order  to  guide  committees  in  identifying  biased  items  that  should  be  examined  more  closely  for 
possible  bias.  Sample  sizes  were  large  enough  to  compare  male  and  female  students,  White  and  Black 
students,  and  White  and  Hispanic  students. 

The  DIF  analyses  of  the  dichotomous  items  were  based  on  the  Mantel-Haenszel  chi-square 
procedure,  as  adapted  by  Holland  and  Thayer  (1988),  which  is  described  in  Chapter  9.  The  procedure 
tests  the  statistical  hypothesis  that  the  odds  of  correctly  answering  an  item  are  the  same  for  two  groups  of 
examinees  that  have  been  matched  on  some  measure  of  mathematics  ability  (usually  referred  to  as  the 
matching  criterion).  The  DIF  analyses  of  the  polytomous  items  were  based  on  the  Mantel  procedure 
(1963)  and  the  Somes  (1986)  chi-square  test  (see  also  Chapter  9).  These  procedures  compare  proportions 
of  matched  examinees  from  each  group  in  each  polytomous  item  response  category.  The  groups  being 
compared  are  often  referred  to  as  the  focal  group  (usually  a minority  or  other  group  of  interest,  such  as 
Black  examinees  or  female  examinees)  and  the  reference  group  (usually  White  examinees  or  male 
examinees). 

For  each  dichotomous  item  in  the  assessment,  an  estimate  was  produced  of  the  Mantel-Haenszel 
common-odds  ratio,  expressed  on  the  ETS  delta  scale  for  item  difficulty.  The  estimates  indicate  the 
difference  between  reference  group  and  focal  group  item  difficulties  (measured  in  ETS  delta  scale  units), 
and  typically  run  between  about  -i-3  and  -3.  Positive  values  indicate  items  that  are  differentially  easier  for 
the  focal  group  than  the  reference  group  after  making  an  adjustment  for  the  overall  level  of  mathematics 
ability  in  the  two  groups.  Similarly,  negative  values  indicate  items  that  are  differentially  harder  for  the 
focal  group  than  the  reference  group.  It  is  common  practice  at  ETS  to  categorize  each  item  into  one  of 
three  categories  (Peterson,  1988):  “A”  (items  exhibiting  no  DIF),  “B”  (items  exhibiting  a weak  indication 
of  DIF),  or  “C”  (items  exhibiting  a strong  indication  of  DIF).  Items  in  category  “A”  have  Mantel- 
Haenszel  common-odds  ratios  on  the  delta  scale  that  do  not  differ  significantly  from  0 at  the  alpha  = 0.05 
level  or  are  less  than  1.0  in  absolute  value.  Category  “C”  items  are  those  with  Mantel-Haenszel  values 
that  are  significantly  greater  than  1 and  larger  than  1.5  in  absolute  magnitude.  Other  items  are  categorized 
as  B items.  A plus  sign  (-I-)  indicates  that  items  are  differentially  easier  for  the  focal  group;  a minus  sign 
(-  ) indicates  that  items  are  differentially  more  difficult  for  the  focal  group. 

The  ETS/NAEP  DIF  procedure  for  polytomous  items  incorporates  both  the  MH  ordinal 
procedure  and  the  generalized  MH  statistic.  The  summary  tables  of  identified  polytomous  items  contain 
generalizations  of  the  dichotomous  “A,”  “B,”  and  “C”  categories  namely  “AA,”  “BB,”  and  “CC” 
respectively.  Analogous  to  the  dichotomous  case,  only  the  “CC”  items  are  considered  to  have  a strong 
indication  of  DIF  and  are  flagged  for  scrutiny  by  the  subject  matter  committee 

For  each  block  of  items  at  each  grade  four  DIF  comparisons  were  made:  male/female, 
White/Asian  American,  White/Black,  and  White/Hispanic.  The  first  subgroup  in  each  comparison  is  the 
reference  group;  the  second  subgroup  is  the  focal  group. 

Following  standard  practice  at  ETS  for  DIF  analyses  conducted  on  final  test  forms,  all 
“C”  and  “CC”  items  were  reviewed  by  a committee  of  trained  test  developers  and  subject-matter 
specialists.  As  indicated  by  Tables  12-14  and  12-15,  two  dichotomous  items  and  five  polytomous 
items  met  the  criteria  to  be  considered  by  the  DIF  committee.  Such  committees  are  charged  with 
making  judgments  about  whether  or  not  the  differential  difficulty  of  an  item  is  unfairly  related  to 
group  membership.  See  Appendix  J for  a list  of  the  “C”  and  “CC”  items. 


•) 
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The  committee  assembled  to  review  NAEP  items  included  both  ETS  staff  and  outside  members 
with  expertise  in  the  field.  It  was  the  committee’s  judgment  that  none  of  the  “C”  or  “CC”  items  for  the 
national  data  were  functioning  differentially  due  to  factors  irrelevant  to  test  objectives.  In  other  words,  all 
of  the  items  that  were  classified  as  “C”  or  “CC”  items  measured  concepts  in  the  assessment  framework 
and  specifications  that  could  not  be  measured  in  another  way.  Hence,  none  of  the  items  were  removed 
from  scaling  due  to  differential  item  functioning. 


Table  12-14 

DIF  Category  by  Grade  for  Dichotomous  Items 

DIF 

Analysis 

Grade 

Category’ 

Male/Female 

White/Black 

White/Hispanic 

4 

C- 

0 

0 

0 

B- 

1 

2 

0 

A- 

11 

9 

10 

A+ 

7 

5 

9 

B+ 

1 

4 

1 

C+ 

0 

0 

0 

8 

C- 

0 

0 

0 

B- 

2 

0 

0 

A- 

13 

7 

13 

A+ 

9 

15 

8' 

B+ 

0 

2 

3 

C+ 

0 

0 

0 

12 

C- 

0 

0 

0 

B- 

3 

1 

0 

A- 

14 

8 

11 

A+ 

5 

12 

10 

B+ 

0 

0 

0 

C+ 

0 

1 

1 

rosiiive  values  oi  me  iiiuca  utiuis  mtii  ^ ^ 

Black,  or  Hispanic  students)  than  for  the  reference  groups  (male  or  White  students).  “A+”  or  “A-” 
means  no  indication  of  DIF,  “B+”  means  a weak  indication  of  DIF  in  favor  of  the  focal  group,  B- 
means  a weak  indication  of  DIF  in  favor  of  the  reference  group  and  “C+”  or  “C-”  means  a strong 
indication  of  DIF. 
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Table  12-15 

DIF  Category  by  Grade  for  Polytomous  Items 


Grade 

DIF 

Category* 

Male/Female 

Analysis 

White/Black 

White/Hispanic 

4 

CC- 

0 

0 

0 

BB- 

1 

1 

0 

AA- 

13 

14 

12 

AA+ 

10 

11 

15 

BB+ 

3 

1 

0 

CC+ 

0 

0 

0 

8 

CC- 

0 

3 

1 

BB- 

0 

2 

2 

AA- 

10 

14 

13 

AA+ 

16 

7 

10 

BB+ 

1 

1 

1 

CC+ 

0 

0 

0 

12 

CC- 

0 

0 

0 

BB- 

2 

3 

1 

AA- 

. 9 

14 

15 

AA+ 

18 

10 

12 

BB+ 

1 

2 

2 

CC+ 

0 

1 

0 

— , ivi  uit;  v^iciimic, 

Black,  or  Hispanic  students)  than  for  the  reference  groups  (male  or  White  students).  “A+”  or  “A-” 
means  no  indication  of  DIF,  “B+”  means  a weak  indication  of  DIF  in  favor  of  the  focal  group,  “B-“ 
means  a weak  indication  of  DIF  in  favor  of  the  reference  group  and  “C+”  or  “C-”  means  a strong 
indication  of  DIF. 


12.5.3  Estimation  of  Item  Parameters 

The  BILOG/PARSCALE  computer  program  was  used  to  estimate  the  item  parameters  for  the  main 
assessment  and  for  the  special  estimation  study.  For  dichotomous  multiple-choice  and  dichotomized 
constructed-response  items,  a three-parameter  IRT  model  was  used.  Extended  constructed-response  and 
cluster  items  were  polytomously  scored  and  were  analyzed  with  a generalized  partial  credit  model  (Muraki, 


Recall  from  section  12.5  that  for  calibration,  items  that  were  missing  prior  to  the  last  completed 
item  in  a block  were  considered  omitted  and  scored  as  wrong.  Also,  items  that  were  not  reached  were 
treated  as  if  they  were  not  presented  to  the  examinees  (and  not  counted  as  wrong.)  However,  there  is  an 
exception  for  blocks  that  end  with  an  extended  constructed-response  item.  In  these  blocks,  students  who 
respond  to  the  next-to-last  item  but  do  not  respond  to  the  last  item  are  classified  as  having  omitted  the  last 
item  (i.e.,  the  last  item  is  counted  as  wrong).  Responses  to  extended  constructed-response  items  that  were 
off-task  were  also  treated  as  omitted.  The  multi-category  constructed-response  items  had  two,  three,  four  or 
five  categories  of  partial  credit.  Scoring  levels  were  labeled  as  listed  in  Table  12-16. 


O 
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Table  12-16 

Labels  for  Score  Levels  ofPolytomous  Items 


Score 

3-Categorv  Item 

4-Categorv  Item 

5-Category  Item 

4 

Correct 

3 

Correct 

4*  Category 

2 

Correct 

3*^  Category 

3"*  Category 

1 

Category 

2"^^  Category 

2"'^  Category 

0 

Incorrect,  off-task,  or 

Incorrect,  off-task,  or 

Incorrect,  off-task,  or 

omitted 

omitted 

omitted 

Note  that  the  categories  falling  between  “incorrect”  and  “correct”  represent  increasing  levels  of  a partially 
correct  response. 

The  item  parameter  estimation  was  done  separately  within  grade,  but  the  final  mathematics  scale 
estimates  were  transformed  to  conform  with  the  cross-grade  scales  of  the  1990  and  1992  assessments. 

Within  each  grade,  items  were  scaled  using  the  grade-only  sample  of  students  available  from  the  1992  and 
the  1996  assessments.  The  1992  data  were  included  at  the  scaling  step  of  the  analysis  to  assure  that 
parameter  estimates  for  items  administered  at  both  time  points  would  not  drift  drastically  between 
assessments.  Item  parameters  were  estimated  separately  for  each  of  the  five  content  strands  and  the 
mathematics  scale  means  and  variances  for  samples  from  the  two  assessment  years  were  allowed  to  be 
different.  In  the  final  BILOG/PARSCALE  mn,  the  prior  distributions  of  the  population  abilities  were  free  to 
be  estimated  and  the  overall  distribution  was  centered  at  zero.  In  general,  if  an  item  was  common  to  both 
assessment  years,  identical  item  parameters  were  used  for  both  assessments.  The  appropriateness  of  the  use 
of  the  identical  parameters  across  assessments  was  examined  by  comparing  the  fit  of  the  empirical  item 
response  functions  against  the  estimated  IRT  item  response  functions.  If  IRT  parameters  did  not  fit  the  data, 
parameters  specific  to  the  assessment  year  were  used.  (See  Chapter  1 1 for  further  descriptions  of  the  scaling 
process.)  The  calibration  was  based  on  student  weights  that  were  rescaled  so  that  the  their  sum  equaled  the 
unweighted  sample  size  of  the  1996  sample.  Also,  weights  for  the  1992  data  were  restandardized  to  give 
equal  weight  to  the  two  assessment  years  included  in  the  scaling  (see  Appendix  K). 

Items  that  received  special  treatment  in  the  scaling  procedure  are  listed  in  Table  12-17,  along  with 
the  reason  for  special  treatment.  Items  were  dropped,  combined  into  clusters,  split  between  assessment  years 
and  collapsed.  If  items  had  empirical  item  response  functions  that  were  nonmonotonic,  they  were  dropped. 

If  several  items  had  highly  correlated  responses  (conditional  on  theta)  they  were  combined  into  a single 
polytomous  item  called  a cluster  item.  If  items  were  administered  in  both  1992  and  1996  but  showed 
evidence  of  having  a distinct  item  response  functions  for  each  assessment  year,  the  item  is  treated  as  two 
separate  items  and  parameters  estimated  separately  for  each  assessment  year.  If  polytomous  items  had 
sparse  or  nonmonotonic  responses  in  one  or  more  categories,  the  items  were  collapsed  so  that  some 
response  categories  were  combined  into  a single  category.  Only  about  ten  percent  of  the  total  scaled  items 
were  given  special  treatment. 
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Table  12-17 

Items  from  the  1996  Assessment  in  Mathematics  Receiving  Special  Treatment 


Grade 

NAEP 

ID 

Block 

Content 

Strand  Treatment 

4 

M010531 

M8 

1 

1992  and  1996  responses  split  into  items 
M010531Y  and  M010531Z 

4 

M040801 

M9 

2 

1992  and  1996  responses  split  into  items 
M040801 Y and  M040801Z 

4 

M041001 

M9 

2 

1992  and  1996  responses  split  into  items 
M041001Y  and  M041001Z 

4 

M041201 

M9 

3 

1992  and  1996  responses  split  into  items 
M041201Y  and  M041201Z 

Categories  in  both  items  collapsed:  0,1, 2, 3, 4 becomes 
0,1,2,3,3, 

4 

M068001 

M7 

1 

Collapsed  categories:  0,1,2  becomes  0,0,1 

4 

M068003 

M7 

3 

Collapsed  categories:  0,1,2  becomes  0,0,1 

4 

M072701 

M14 

5 

Collapsed  categories:  0,1,2,3,4  becomes  0,1,2,3,3 

4 

M074701 

M15 

1 

Collapsed  categories:  0,1,2  becomes  0,0,1 

8 

M013531 

M8 

1 

1992  and  1996  responses  split  into  items 
M013531Y  and  M013531Z 

8 

MO  18201 

M4 

1 

1992  and  1996  responses  split  into  items 
M018201Y  and  M018201Z 

8 

MO  18901 

M4 

4 

1992  and  1996  responses  split  into  items 
M018901Y  and  M018901Z 

8 

M045901 

1992 

3 

Collapsed  categories:  0,1, 2,3,4  becomes  0,1, 2,3,3 
1992  only 

8 

M050261 

M3 

4 

Collapsed  categories:  0,1, 2,3,4  becomes  0,0,0, 1,2 

8 

M051001 

M3 

3 

1992  and  1996  responses  split  into  items 
M051001Y  and  M051001Z 

8 

M051101 

M3 

1 

1992  and  1996  responses  split  into  items 
M05 1 101 Y and  M05 1 lOlZ 

Categories  in  both  items  collapsed:  0,1,2,3,4  becomes 
0,1,2,3,3 

8 

M051201 

M13 

1 

1992  and  1996  responses  split  into  items 
M051201Y  and  M051201Z 

8 

M052201 

M13 

2 

Collapsed  categories:  0,1, 2,3,4  becomes  0,1, 2,3,3 

8 

M053101 

M9 

4 

Collapsed  categories:  0,1, 2,3,4  becomes  0,1, 2,2,3 

8 

M054301 

1992 

5 

Collapsed  categories:  0,1, 2,3,4  becomes  0,1, 2,3,3 
1992  only 

'J  V 
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Table  12-17  (continued) 

Items  from  the  1996  Assessment  in  Mathematics  Receiving  Special  Treatment 


Grade 

NAEP 

ID 

Block 

Content 

Strand 

Treatment 

8 

M055501 

1992 

1 

Collapsed  categories:  0,1, 2, 3, 4 becomes  0,1,2,3,3 
1992  only 

8 

M066301 

M5 

5 

Collapsed  categories:  0,1,2  becomes  0,1,1 

8 

M067501 

M5 

4 

Collapsed  categories:  0,1, 2, 3,4  becomes  0,1, 2,2,2 

8 

M068201 

M7 

3 

Collapsed  categories:  0,1, 2, 3,4  becomes  0,0, 1,1,1 

8 

M069301 

M12 

5 

Collapsed  categories:  0,1,2  becomes  0,1,1 

8 

M069601 

M12 

1 

Collapsed  categories:  0,1,2  becomes  0,1,1 

8 

M070001 

M12 

4 

Collapsed  categories:  0,1, 2,3  becomes  0,1, 2,2 

8 

M0732CL 

M14 

4 

Collapsed  categories:  0,1, 2,3, 4 becomes  0,0, 1,2,3 

8 

M073401 

M14 

1 

Dropped  due  to  bad  item  fit 

8 

M073501 

M14 

4 

Collapsed  categories:  0,1,2  becomes  0,1,1 

8 

M073601 

M14 

1 

Collapsed  categories:  0,1,2,3,4  becomes  0,1,2,3,3 

8 

M0757CL 

M15 

5 

Collapsed  categories:  0,1, 2,3  becomes  0,0, 1,2 

8 

M076001 

M15 

2 

Collapsed  categories:  0,1,2,3,4  becomes  0,1,2,3,3 

12 

M073402 

M14 

1 

Dropped 

12 

M025301 

M5 

1 

Dropped  1992  only 

12 

M024901 

M5 

2 

Dropped  1992  only 

12 

M070501, 
M070502  ' 

M12 

4 

Combined  into  cluster  item  M0705CL 

12 

M071701, 

M071702 

M7 

5 

Combined  into  cluster  item  M0717CL 

12 

M071401 

M5 

1 

Collapsed:  0,1,2,3,4  becomes  0,1, 1,1,2 

12 

M056601 

M3 

1 

1992  and  1996  responses  split  into  items 
M056601Y  and  M056601Z 

12 

M062401 

MIO 

3 

1992  and  1996  responses  split  into  items 

M062401Y  and  M062401Z 

Both  items  collapsed:  0,1,2,3,4  becomes  0,1,2,3,3 

12 

M073901 

M14 

3 

Collapsed  categories:  0,1,2  becomes  0,1,1 

12 

M012731 

M8 

3 

1992  and  1996  responses  split  into  items 
M012731YandM012731Z 

12 

M058701 

Mil 

3 

1992  and  1996  responses  split  into  items 
M058701Y  and  M058791Z 

12 

M071801 

M7 

4 

Collapsed  categories:  0,1, 2, 3,4  becomes  0,1, 1,2, 3 
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Table  12-17  (continued) 

Items  from  the  1 996  Assessment  in  Mathematics  Receiving  Special  Treatment 


NAEP 

Content 

Grade 

ID 

Block 

Strand 

Treatment 

12 

M070601 

M12 

4 

Collapsed  categories:  0,1,2,3,4  becomes  0,1,2,3,3 

12 

M0732CL 

M14 

4 

Collapsed  categories:  0,1, 2,3,4  becornes  0,0, 1,2,3 

12 

MO  18901 

M4 

4 

1992  and  1996  responses  split  into  items 
M018901YandM018901Z 

12 

M021501 

M6 

4 

1992  and  1996  responses  split  into  items 
M021501Y  and  M021501Z 

12 

M013131 

M8 

4 

1992  and  1996  responses  split  into  items 
M013131Y  and  M013131Z 

12 

M053401 

M9 

. 4 

1992  and  1996  responses  split  into  items 
M05340 1 Y and  M05340 1 Z 

12 

M061905 

MIO 

4 

1992  and  1996  responses  split  into  items 
M061905Y  and  M061905Z 

12 

M060701 

M13 

5 

1992  and  1996  responses  split  into  items 
M060701Y  and  M060701Z 

Both  items  collapsed:  0,1, 2,3,4  becomes  0,1, 2,3,3 

12 

M074101 

M14 

5 

Collapsed  categories:  0,1, 2,3,4  becomes  0,1, 2,3,3 

12 

M077001 

M15 

5 

Collapsed  categories:  0,1, 2,3,4  becomes  0,1, 1,1,1 

12 

M021601 

M6 

5 

1992  and  1996  responses  split  into  items 
M021601Y  and  M021601Z 

12 

M021602 

M6 

5 

1992  and  1996  responses  split  into  items 
M021602Y  and  M021602Z 

12.5.4  Evaluating  the  Fit  of  the  IRT  Model 


During  the  course  of  estimating  an  IRT  model,  individual  items  were  evaluated  to  determine  how 
well  the  item  response  model  fit  the  data.  This  was  done  by  visual  inspection  of  plots  comparing 
empirically  based  and  theoretical  item  response  functions.  Specifically,  for  dichotomous  items  these  plots 
consisted  of  nonmodel-based  estimates  of  the  expected  proportion  correct  for  each  level  of  mathematics 
ability  compared  to  the  proportion  correct  for  each  level  of  mathematics  ability  as  predicted  by  the 
theoretical  item  response  function.  For  polytomous  extended  constructed-response  items,  similar  plots 

were  produced  for  each  item  category  response  function  (see  Chapter  9 for  a fuller  explanation  of  these 
plots). 


In  making  decisions  about  excluding  items  from  the  final  scales,  a balance  was  sought  between 
being  too  stringent,  hence  deleting  too  many  items  and  possibly  damaging  the  content  representativeness 
of  the  pool  of  scaled  items,  and  being  too  lenient,  hence  including  items  with  model  fit  poor  enough  to 
endanger  the  types  of  model-based  inferences  made  from  NAEP  results.  Items  that  clearly  did  not  fit  the 


model  were  not  included  in  the  final  scales;  however,  a certain  degree  of  misfit  was  tolerated  for  a 
number  of  items  included  in  the  final  scales. 

For  most  items,  the  model  fit  well.  In  a few  cases,  poor  fit  with  the  data  led  to  special  treatment 
or  deletion  of  the  item.  Figures  12-1,  12-4,  12-5  and  12-6  give  item  response  plots  of  dichotomous  items. 
In  the  plots,  the  x-axis  depicts  scale  score,  and  the  y-axis  the  probability  of  a correct  response.  The  solid 
line  is  the  logistic  model  prediction,  and  the  symbols  (usually  a circle  or  diamond)  are  the  nonmodel- 
based  predicted  proportions.  The  size  of  the  symbols  are  proportional  to  the  estimated  number  of  students 
at  a particular  scale  score  level.  The  symbols  are  ordinarily  larger  in  the  middle  of  the  theta  scale,  where 
most  students’  scale  scores  fall.  The  item  parameter  values  are  also  included  in  the  plot. 

Item  response  plots  for  polytomously  scored  items  are  given  in  Figures  12-2,  12-3,  12-7,  and 
12-8.  These  are  similar  to  the  plots  for  dichotomous  items  except  that  there  are  several  solid  lines,  one  for 
each  item  category,  with  each  line  indicating  the  probability  of  responding  in  the  respective  item 
category.  As  before,  the  circles  or  diamonds  indicate  the  empirical  response  function,  with  the  size  of  the 
circles  and  diamonds  proportional  to  the  estimated  number  of  students  at  a scale  score  level. 

In  the  plots  good  fit  of  the  model  to  the  data  is  indicated  when  the  model-based  functions  (solid 
lines)  coincide  with  the  empirical  functions  (circles,  diamonds,  or  other  symbols).When  the  empirical 
plot  is  far  away  from  the  model  based  line,  there  is  poor  fit  of  the  model  to  the  data. 

Four  examples  of  fit  are  illustrated.  First  there  is  good  model  fit  which  is  shown  by  Figure  12-1 
for  a dichotomous  item  and  Figure  12-2  for  a polytomous  item.  In  both  cases  empirical  and  theoretical 
lines  coincide. 

Second  is  an  example  of  an  item  that  displayed  non-logistic  empirical  functions  and  was 
dropped.  Figure  12-3  shows  a polytomous  item  that  was  dropped  from  the  assessment. 

Third  is  an  example  of  an  item  that  changed  function  from  one  assess  year  to  another.  Figure  12- 
4 shows  that  the  empirical  item  functions  for  two  assessment  years  (diamonds  for  1992  and  circles  for 
1996)  are  distinctly  different.  Figures  12-5  and  12-6  show  the  result  of  estimating  item  parameters 
separately  for  the  two  years.  This  ‘splitting’  of  the  item  across  years  results  in  quite  good  fit  for  each 
year. 

The  fourth  example  is  of  a poor  fitting  polytomous  item  that  was  modified  by  collapsing 
categories.  Figure  12-7  shows  a 5-category  item  which  evidences  poor  fit  mostly  in  the  upper  category, 
due  in  large  part  to  a low  number  of  respondents^  As  a result,  the  upper  two  categories  were  collapsed 
resulting  in  a 4-category  item  which,  as  Figure  12-8  illustrates,  fits  satisfactorily. 
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Figure  12-1 

Dichotomous  Item  Exhibiting  Good  Model  Fit* 


♦Circles  (1996  data)  and  diamonds  (1992  data)  indicate  estimated  conditional  probabilities  obtained  without  assuming  a logistic 
form;  the  solid  curve  indicates  estimated  item  response  function  assuming  a logistic  form. 
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Figure  12-2 

Polytomous  Item  Exhibiting  Good  Model  Fit* 


NAEP  ID:  M075101  SEQUENCE:  060  SCALE:  NUM&OPER 


♦Circles  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the  solid  curves  indicate 
estimated  item  response  function  assuming  a model-based  form. 
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Figure  12-3 

Polytomous  Item  Exhibiting  Bad  Model  Fit  That 
Was  Deleted  from  the  Assessment* 


*Circles  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the  solid  curves  indicate 
estimated  item  response  function  assuming  a model-based  form. 
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Figure  12-4 

Dichotomous  Item  Exhibiting  Different  Empirical  Item  Functions  for  Different  Assessment  Years* 


NAEP  ID;  M018901 


SCALE:  DATA  ANL 


♦Diamonds  represent  1992  data;  circles  represent  1996  data.  Circles  and  diamonds  indicate  estimated  conditional  probabilities 
obtained  without  assuming  a logistic  form;  the  solid  curve  indicates  estimated  item  response  function  assuming  a logistic  form. 
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Figure  12-5 

Dichotomous  Item  Fit  Separately  to  the  1992  Data  and  Exhibiting  Good  Model  Fit* 


NAEPID:  M018901Y  SCALE:  DATA  ANL 


o 


♦Diamonds  indicate  estimated  conditional  probabilities  obtained  without  assuming  a logistic  form;  the  solid  curve  indicates 
estimated  item  response  function  assuming  a logistic  form. 
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Figure  12-6 

Dichotomous  Item  Fit  Separately  to  the  1996  Data  and  Exhibiting  Good  Model  Fit* 

NAEP  ID:  M018901Z  • SCALE:  DATA  ANL 


♦Circles  indicate  estimated  conditional  probabilities  obtained  without  assuming  a logistic  form;  the  solid  curve  indicates 
estimated  item  response  function  assuming  a logistic  form. 
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Figure  12-7 

Polytomous  Item  Exhibiting  Poor  Model  Fit  in  the  Upper  Category* 


THETA 


♦Circles  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the  solid  curves  indicate 

estimated  item  response  function  assuming  a model-based  form. 
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Figure  12-8 

Polytomous  Item  With  the  Upper  Two  Categories  Collapsed,  Now  Exhibiting  Good  Model  Fit* 


-4100  -3.0  -2.(D  -1.®  0.0  1.®  2.0  3.0  4.0 

THETA 


♦Circles  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the  solid  curves  indicate 
estimated  item  response  function  assuming  a model-based  form. 


12.5.5  Derived  Background  Variables 

Derived  variables  are  variables  which  use  information  from  more  than  one  background  question. 
They  were  used  for  two  purposes:  as  conditioning  variables  and  as  reporting  variables  used  to  define 
subgroups.  Some  of  these  variables  are  common  to  all  the  subject  areas;  others  are  specific  to  the  1996 
mathematics  assessment.  Derived  variables  used  for  conditioning  and  reporting  are  described  in  Appendix  C. 


311 


283 


12.5.6  Generation  of  Plausible  Values 


For  the  entire  sample,  multivariate  plausible  values  for  content  strand  scales  were  generated  for 
each  grade  group  separately  using  the  multivariate  conditioning  program  CGROUP  as  revised  by  Thomas 
(1993).  As  with  the  scaling,  student  weights  were  used  at  this  stage  of  the  analysis.  Instead  of  using  selected 
background  variables  for  conditioning  variables  (as  had  been  done  prior  to  the  1990  assessment),  principal 
components  of  the  background  variables  were  used.  The  principal  components  used  accounted  for  90 
percent  of  the  variance  of  the  original  conditioning  variables.  Principal  components  were  employed  to 
remedy  problems  of  extreme  collinearity  among  some  of  the  original  conditioning  variables. 

Research  based  on  data  from  the  1990  Trial  State  Assessment  in  mathematics  suggests  that  results 
obtained  using  the  90  percent  subset  of  components  will  differ  only  slightly  from  those  obtained  using  the 
full  set  (Mazzeo,  Johnson,  Bowker,  & Fong,  1992).  Table  12-18  lists  the  number  of  principal  components 
included  in  conditioning,  as  well  as  the  proportion  of  variance  accounted  for  by  the  conditioning  model  for 
each  grade. 


Table  12-18 

Proportion  of  Scale  Score  Variance  Accounted  for  by  the  Conditioning  Model 
for  the  Mathematics  Main  Assessment 


Proportion  of  Scale  Score  Variance 


Grade 

Number  of 
Conditioning 
Contrasts 

Number  of 
Principal 
Components 

Number 

Sense, 

Properties, 

and 

Operations 

Measurement 

Geometry 

and 

Spatial 

Sense 

Data  ’ 
Analysis, 
Statistics, 
and 

Probability 

Algebra 

and 

Functions 

4 

895 

321 

.70 

.71 

.61 

.75 

.69 

8 

1,027 

362 

.71 

.75 

.68 

.77 

.73 

12 

812 

314 

.76 

.79 

.81 

.70 

.83 

The  codings  of  the  original  mathematics-specific  conditioning  variables,  before  principal 
components  were  calculated,  are  presented  in  Appendix  C.  The  CGROUP  program  estimates  distributions 
of  scale  scores  by  combining  information  from  item  responses  of  individuals  and  information  from  linear 
regression  of  scale  score  on  conditioning  variables.  For  each  individual,  five  plausible  values  are  randomly 
drawn  from  their  predictive  conditional  distribution. 

The  proportion  of  variance  of  each  original  conditioning  variable  accounted  for  by  the  principal 
components  included  in  the  conditioning  model  is  listed  in  Appendix  F.  The  estimated  conditioning  effects 
for  the  principal  components  of  the  three  samples  defined  by  the  three  grade  groups  are  also  given  in 
Appendix  C.  The  values  of  the  conditioning  effects  are  expressed  in  the  metrics  of  the  original  calibration 
scale.  Definitions  of  derived  conditioning  variables  are  given  in  Appendix  B. 
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12.5.7  The  Transformation  of  the  Mathematics  Calibration  Scale  for  Reporting 
and  the  Formation  of  the  Composite  Scale 

Like  all  IRT  scales,  the  mathematics  content  strand  scales  have  a linear  indeterminacy  that  may  be 
resolved  by  an  arbitrary  choice  of  the  origin  and  unit-size  in  each  given  content  strand.  In  1990  the  NAEP 
mathematics  data  were  scaled  across  grades  separately  for  each  scale.  The  linear  indeterminacies  among  the 
scales  were  resolved  by  transforming  the  scale  means  and  variances  of  three  grade  samples  combined 
together  to  the  250.5  50.0  metric  using  a transformation  of  the  form, 

^target  “ A • Ocalibrated 

where  A and  B are  linear  transformation  constants. 

As  a result,  all  of  the  scales  that  spanned  all  three  grade  samples  were  on  a common  scale.  By 
contrast,  the  1992  and  1996  data  were  scaled  within  grade.  It  was  necessary,  therefore,  to  transform  data 
from  both  assessments  to  the  1990  cross-grade  scale.  This  was  accomplished  first  in  the  1992  assessment, 
when  the  1992  data  were  linked  to  the  1990  scale  in  a two-stage  process.  In  the  next  assessment,  the  1996 
data  were  linked  to  the  1992  transformed  scale,  which  in  effect  put  the  1996  data  on  the  1990  cross-grade 
scale.  The  procedure  for  transforniing  the  1996  data  will  be  described  below.  The  similar  procedure  for 
transforming  the  1992  data  was  presented  in  The  NAEP  1992  Technical  Report  (Johnson  and  Carlson, 
1994). 

The  1996  data  were  put  on  the  1992  reporting  metric  by  using  a linear  transformation  that  converts 
the  1996  thetas  to  the  1992  reporting  scale.  This  linear  transformation  was  created  by  the  following 
procedure.  The  1992  thetas  were  reconditioned  using  CGROUP  with  the  1996  item  parameters.  This 
analysis  resulted  in  1992  scores  which  were  in  the  1996  theta  metric.  We  then  transformed  the  new  1992 
thetas  (in  the  1996  metric)  to  the  1992  reporting  metric  (in  the  1992  metric)  by  a linear  transformation 
which  created  scores  having  the  same  mean  and  variance  as  the  1992  reporting  scale.  This  is  a common 
population  equating  procedure.  The  linear  constants  of  this  transformation  were  then  used  to  transform  the 
1996  thetas  to  the  1992  reporting  metric.  The  transformation  constants  used  for  the  five  content  strand 
scales  and  for  the  estimation  scale  are  given  in  Table  12-19. 

While  scores  in  five  content  strands  provide  useful  insights  into  the  relationships  among 
subpopulations,  a single  index  to  summarize  overall  performance  is  a useful  tool  for  a compact  overview  of 
subpopulation  trends.  For  that  reason,  a composite  score  was  defined  as  a weighted  average  of  the  five 
mathematics  content  strands.  The  weight  given  to  each  content  strand  is  a direct  reflection  of  the  relative 
testing  time  intended  for  that  content  strand  in  the  assessment,  as  defined  in  the  Mathematics  Framework 
for  the  1996  National  Assessment  of  Educational  Progress  (National  Assessment  Governing  Board,  1994). 
Since  the  emphasis  given  to  each  content  strand  was  different  across  grades,  the  weights  assigned  to  each 
strand  in  the  composite  also  differed  across  grades. 


■*  The  function  RP,  = 250  + 50(0,,)  would  have  been  preferable.  Holland  and  Zwick  (1986)  have  noted  that  the  values  actually 
used  correspond  to  the  bi  varying  from  -5.00  to  +4.98  in  steps  of  .02  instead  of  -4.99  to  +4.99  as  intended.  The  result  is  that  the 
RP  scores  are  a half-point  higher  than  appropriate  for  the  hypothetical  test. 
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Table  12-19 

Coefficients  of  the  Linear  Transformations  that  Transform  the  Five  Content  Strand  Scales 
from  the  1996  Calibration  Metric  to  the  1992  Reporting  Metric 


Grade 

CoefRcient 

Number  Sense, 
Properties, 
and 

Operations 

Measurement 

Geometry 
and  Spatial 
Sense 

Data  Analysis, 
Statistics,  and 
Probability 

Algebra  and 
Functions 

4 

B 

220.32 

226.02 

224.70 

223.38 

223.63 

A 

33.13 

32.22 

28.04 

30.86 

29.47 

8 

B 

273.44 

269.33 

266.83 

270.97 

271.13 

A 

35.12 

43.65 

33.54 

41.00 

35.52 

12 

B 

300.43 

300.85 

304.67 

301.03 

303.42 

A 

32.48 

35.71 

36.05 

34.58 

35.95 

The  definition  of  weights  for  the  composite  in  each  grade  is  given  in  Table  12-20.  The  mean  and 
standard  deviations  of  the  composite  scales  for  all  three  grades  is  given  in  Table  12-21 . Note  that  this 
composite  can  be  compared  with  the  1990  and  1992  composite  scores  since  all  three  scales  are  on  the  1990 
cross-grade  scale  (as  defined  in  the  beginning  of  this  section). 


Table  12-20 

Weights  for  the  Mathematics  Composite  by  Grade 


Scale 

Grade  4 

Grade  8 

Grade  12 

Number  Sense,  Properties,  and  Operations 

.40 

.25 

.20 

Measurement 

.20 

.15 

.15 

Geometry  and  Spatial  Sense 

.15 

.20 

.20 

Data  Analysis,  Statistics,  and  Probability 

.10 

.15 

.20 

Algebra  and  Functions 

.15 

.25 

.25 

Table  12-21 

Means  and  Standard  Deviations  on  the  Mathematics  Composite  Scale 


Grade 

All  Five  Plausible  Values 
Mean  S.  D. 

4 

223.9 

31.2 

8 

272.0 

36.4 

12 

304.0 

32.2 

86 
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12.5.8  Partitioning  of  the  Estimation  Error  Variance 


For  each  scale  within  each  grade,  the  error  variance  of  the  reporting  scale  means  was  partitioned 
according  to  the  procedure  described  in  Chapter  1 1.  The  variance  is  partitioned  into  two  parts;  the 
proportion  of  error  variance  due  to  sampling  students  (sampling  variance)  and  the  proportion  of  error 
variance  due  to  the  fact  that  scale  score,  0,  is  a latent  variable  that  is  estimated  rather  than  observed.  Table 
12-22  contains  estimates  of  the  total  error  variance,  the  proportion  of  error  variance  due  to  sampling 
students  and  the  proportion  of  error  variance  due  to  the  latent  nature  of  0 (for  stability,  the  estimates  of  the 
between-imputation  variance,  B,  in  Equation  1 1.9).  More  detailed  information  by  gender  and  race/ethnicity 
is  presented  in  Appendix  E. 


Table  12-22 

Estimation  Error  Variance  and  Related  Coefficients  for  the  Mathematics  Main  Assessment 


Proportion  of 

Total 

Variance  Due  to... 

Estimation 

Student 

Latency 

Grade 

Scale 

Error  Variance 

Sampling 

ofO 

4 

Number  Sense,  Properties,  and  Operations 

.88 

.91 

.09 

Measurement 

1.25 

.84 

.16 

Geometry  and  Spatial  Sense 

.77 

.82 

.18 

Data  Analysis,  Statistics,  and  Probability 

1.28 

.87 

.13 

Algebra  and  Functions 

.99 

.83 

.17 

Composite 

.78 

.95 

.05 

8 

Number  Sense,  Properties,  and  Operations 

1.18 

.91 

.09 

Measurement 

2.06 

.87 

.13 

Geometry  and  Spatial  Sense 

1.10 

.91 

.09 

Data  Analysis,  Statistics,  and  Probability 

2.31 

.92 

.08 

Algebra  and  Functions 

1.19 

.89 

.11 

Composite 

1.13 

.96 

.04 

12 

Number  Sense,  Properties,  and  Operations 

1.27 

.93 

.07 

Measurement 

1.25 

.84 

.16 

Geometry  and  Spatial  Sense 

1.12 

.88 

.12 

Data  Analysis,  Statistics,  and  Probability 

.99 

.91 

.09 

Algebra  and  Functions 

1.33 

.93 

.07 

Composite 

.99 

.98 

.02 

12.5.9  Mathematics  Teacher  Questionnaire 

Teachers  of  fourth-  and  eighth-grade  students  assessed  in  mathematics  were  surveyed.  Variables 
derived  from  the  questionnaire  were  used  in  the  conditioning  models  for  the  grade  4 and  the  grade  8 
samples,  along  with  a variable  that  indicated  whether  a student  record  had  been  matched  with  a teacher 
record,  so  that  means  for  subgroups  defined  by  these  variables  could  be  compared  with  no  bias.  Of  the  6,612 
fourth-grade  students  in  the  main  sample,  6,105  (92%)  were  matched  with  both  parts  of  the  teacher 
questionnaire  and  99  (1.5%)  were  matched  with  only  the  first  part  of  the  questionnaire.  Of  the  7,146  eighth- 
grade  students  in  the  main  sample,  6,144  (86%)  were  matched  with  both  parts  of  the  teacher  questionnaire 
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and  49  (less  than  1%)  were  matched  with  only  the  first  part  of  the  questionnaire.  Thus,  92  percent  of  the 
fourth  graders  and  86  percent  of  the  eighth  graders  were  matched  with  at  least  the  background  information 
about  their  mathematics  teachers. 


12.5.10  Analysis  of  Dimensionality 

Plausible  values  are  drawn  from  the  set  of  five  correlated  content  strands  (see  Section  12.5.6).  For  this 
reason,  it  is  useful  to  inspect  the  correlations  among  the  content  strands  for  evidence  of  multidimensionality. 
Tables  12-23  and  12-24  give  conditional  and  marginal  correlations  for  the  five  scales  for  the  three  grades. 
Conditional  correlations  are  analogous  to  pooled-within  groups  correlations  when  the  groups  are  the  grouping 
variables  used  to  condition  the  data  with  CGROUP.  They  are  obtained  from  the  error  correlations  of  a 
CGROUP  analysis.  The  conditional  correlations  are  quite  high,  averaging  .85  for  grade  4,  .91  for  grade  8,  and 
.85  for  grade  12.  The  marginal  correlations  are  the  average  correlations  of  the  five  plausible  values  of  each 
scale.  In  this  case  they  average  .86  for  grade  4,  .88  for  grade  8,  and  .87  for  grade  12.  Although  it  is  of 
substantive  interest  to  analyze  the  scales  separately,  the  correlations  indicate  that  they  are  highly  redundant. 


Table  12-23 

Conditional  Correlations  from  Conditioning  (CGROUP) 


Number  Sense, 

Geometry 

Data  Analysis, 

Properties, 

and 

Statistics, 

Algebra 

* 

and 

Spatial 

and 

and 

Grade 

Scale 

Operations 

Measurement 

Sense 

Probability 

Functions 

4 

Number  Sense,  Properties,  and 

Operations 

1.00 

Measurement 

.89 

1.00 

Geometry  and  Spatial  Sense 

.80 

.78 

1.00 

Data  Analysis,  Statistics,  and 

Probability 

96 

.90 

.75 

1.00 

Algebra  and  Functions 

.94 

.85 

.78 

.89 

1.00 

8 

Number  Sense,  Properties,  and 

Operations 

1.00 

Measurement 

.92 

1.00 

Geometry  and  Spatial  Sense 

.79 

.92 

1.00 

Data  Analysis,  Statistics,  and 

Probability 

.96 

.95 

.86 

1.00 

Algebra  and  Functions 

.97 

.95 

.84 

.96 

1.00 

12 

Number  Sense,  Properties,  and 

Operations 

1.00 

Measurement 

.96 

1.00 

Geometry  and  Spatial  Sense 

.90 

.94 

1.00 

Data  Analysis,  Statistics,  and 

Probability 

.84 

.80 

.67 

1.00 

Algebra  and  Functions 

.91 

.89 

.88 

.66 

1.00 

316 


Table  12-24 

Marginal  Correlations  of  Science  Scales^ 


Number  Sense, 

Geometry 

Data  Analysis, 

Properties, 

and 

Statistics, 

Algebra 

and 

Spatial 

and 

and 

Grade 

Scale 

Operations 

Measurement 

Sense 

Probability 

Functions 

4 

Number  Sense,  Properties,  and 

Operations 

1.00 

Measurement 

.89 

1.00 

Geometry  and  Spatial  Sense 

.84 

:83 

1.00 

Data  Analysis,  Statistics,  and 

Probability 

.91 

.86 

.81 

1.00 

Algebra  and  Functions 

.92 

:86 

.82 

.86 

1.00 

8 

Number  Sense,  Properties,  and 

Operations 

1.00 

Measurement 

.89 

1.00 

Geometry  and  Spatial  Sense 

.84 

.89 

1.00 

Data  Analysis,  Statistics,  and 

Probability 

.91 

.89 

.85 

1.00 

Algebra  and  Functions 

.92. 

.89 

.86 

.90 

1.00 

12 

Number  Sense,  Properties,  and 

Operations 

1.00 

Measurement 

.90 

1.00 

Geometry  and  Spatial  Sense 

.90 

.90  . 

1.00 

Data  Analysis,  Statistics,  and 

Probability 

. .88 

.86 

.81 

1.00 

Algebra  and  Functions 

.90 

.87 

.90 

.80 

1.00 

‘ Tabled  values  were  obtained  by  computing  a separate  Pearson  correlation  coefficient  for  each  plausible  value,  computing  Fisher’s  z-transformation 
for  each  value,  computing  the  average  of  the  transformed  values,  and  computing  the  inverse  transformation  of  the  average. 
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Chapter  13 


DATA  ANALYSIS  FOR  THE  SCIENCE  ASSESSMENT^ 

John  R.  Donoghue,  Jinming  Zhang,  Steven  P.  Isham, 

Lois  H.  Worthington,  and  Ingeborg  U.  Novatkoski 
Educational  Testing  Service 


13.1  OVERVIEW 

This  chapter  describes  the  analyses  performed  on  the  responses  to  the  cognitive  and  background 
items  in  the  1996  assessment  of  science.  These  analyses  led  to  the  results  presented  in  the  NAEP  1996 
Science  Report  Card  for  the  Nation  and  the  States:  Findings  from  the  National  Assessment  of  Educational 
Progress  (O’Sullivan,  Reese,  & Mazzeo,  1997).  The  emphasis  of  this  chapter  is  on  the  methods  and 
results  of  procedures  used  to  develop  the  IRT-based  scale  scores  that  formed  the  basis  of  these  reports. 
However,  some  attention  is  given  to  the  analysis  of  constructed-response  items  as  reported  in  the  NAEP 
1996  Science  Report  Card  for  the  Nation  and  the  States.  The  theoretical  underpinnings  of  the  IRT  and 
plausible  values  methodology  described  in  this  chapter  are  given  in  Chapter  11,  and  several  of  the 
statistics  are  described  in  Chapter  9. 

For  1996,  the  NAEP  science  assessment  framework  incorporated  a balance  of  knowledge  and 
skills  based  on  current  reform  reports,  exemplary  curriculum  guides,  and  research  on  the  teaching  and 
learning  of  science.  The  1996  assessment  included  the  use  of  hands-on  science  tasks  and  theme  blocks  as 
well  as  considerably  more  constructed-response  items  than  previous  NAEP  assessments. 

The  student  samples  that  were  administered  science  items  in  the  1996  assessment  are  shown  in 
Table  13-1.  Chapters  1 and  3 contain  descriptions  of  the  target  populations  and  the  sample  design  used  for 
the  assessment. 


Table  13-1 

NAEP  1996  Science  Student  Samples 


Sample 

Booklet 

IDs 

Number 

Mode 

Cohort 

Assessed 

Time  of  Testing’ 

Number 

Assessed 

4 [Science  Main] 

201-237 

Print 

Grade  4 

1/3/96  - 3/29/96  (Winter) 

7,305 

8 [Science  Main] 

201-237 

Print 

Grade  8 

1/3/96  - 3/29/96  (Winter) 

7,774 

12  [Science  Main] 

201-237 

Print 

Grade  12 

1/3/96  - 3/29/96  (Winter) 

7,537 

12  [Sci-Advanced] 

238-240 

Print 

Grade  12 

1/3/96  - 3/29/96  (Winter) 

2,431 

‘Final  makeup  sessions  were  held  April  1-5,  1996. 


LEGEND: 

Print  Printed  administration 

Main  Main  assessment 

Advanced  Assessment  with  advanced  booklets 


‘ John  Donoghue  was  the  primary  person  responsible  for  the  planning,  specification,  and  coordination  of  the  science  analyses. 
He  was  assisted  by  Jinming  Zhang.  Computer  activities  for  all  science  scaling  arid  data  analyses  were  directed  by  Steve  Isham 
and  completed  by  Lois  Worthington  and  Ingeborg  Novatkoski.  Others  contributing  to  the  analysis  of  science  data  were  David  S. 
Freund,  Katharine  Pashley,  and  Norma  A.  Norris. 
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The  objectives  of  the  science  analyses  were  to 

® prepare  scale  values  and  estimate  subgroup  proficiency  distributions  for  national 
samples  of  students  who  were  administered  science  items  from  the  main  assessment, 
and 

e prepare  the  analysis  of  the  advanced  science  assessment.  The  advanced  science 
sample  12[Sci-Advanced]  is  a separate  sample  from  the  12[Science  Main]  sample. 

Analyses  of  the  advanced  science  assessment  will  be  described  in  a subsequent 
NAEP  report.  The  12[Sci-Advanced]  sample  is  discussed  further  in  section  13-3. 

The  1996  science  samples  were  analyzed  to  provide  comparisons  of  science  achievement  for 
various  subgroups  of  the  1996  target  populations.  The  target  populations  were  grade  4,  grade  8,  and  grade 
12  students  in  the  United  States.  Unlike  previous  NAEP  assessments,  only  grade-defined  cohorts  were 
assessed  in  the  1996  NAEP.  The  age  of  students  was  based  on  a calendar  year,  with  birthdates  in  1986, 
1982,  and  1978,  respectively,  for  ages  9,  13,  and  17.  The  sampled  students  in  each  of  these  three  cohorts 
were  assessed  in  the  winter  (January  to  March  with  final  makeup  sessions  held  during  the  first  week  of 
April).  As  described  in  Chapter  9,  the  reporting  sample  for  the  national  science  assessment  consisted  of 
students  in  the  S2  sample  (also  see  Chapter  19  for  tables  describing  the  students  assessed  and  the 
reporting  sample  for  each  component  of  the  science  assessment). 

The  major  analysis  components  are  discussed  in  turn.  Some  aspects  of  the  analysis,  such  as 
procedures  for  item  analysis,  scoring  of  constructed-response  items,  and  methods  of  scaling,  are  described 
in  previous  chapters  and  are  therefore  not  detailed  here.  There  were  five  major  steps  in  the  analysis  of  the 
science  data,  each  of  which  is  described  in  a separate  section: 

1.  conventional  item  and  test  analyses,  and  DIF  analyses  (Section  13.4); 

2.  item  response  theory  (IRT)  scaling  (Section  13.5); 

3.  estimation  of  subgroup  proficiency  distributions  based  on  the  “plausible 
values”  methodology  (Section  13.6); 

4.  transforming  the  1996  assessment  scales  to  the  final  reporting  metric  for  each 
of  the  fields  of  science,  and  (Section  13.7.1);  and 

5.  creation  of  the  science  composite  scale  (Section  13.7.2). 

Section  13.8  describes  the  results  of  partitioning  the  error  variance,  13.9  discusses  the  matching  of  student 
responses  to  those  of  their  teachers,  and  13.10  provides  a brief  explanation  of  sampling  weights. 

To  set  the  context  within  which  to  describe  the  methods  and  results  of  scaling  procedures,  a brief 
review  of  the  assessment  instruments  and  administration  procedures  is  provided. 


13.2  DESCRIPTION  OF  ITEMS  AND  ASSESSMENT  BOOKLETS 

The  1996  NAEP  main  science  assessment  differed  from  the  long-term  trend  assessment  in  regard 
to  the  sample  age  definition,  time  of  testing,  the  objectives  that  define  the  emphasis  of  the  assessment,  and 
most  of  the  items  used.  It  also  differed  from  the  1990  main  NAEP  science  assessment  in  the  same  regards. 
Because  of  these  differences,  equating  or  linking  the  main  and  the  long-term  trend  assessments  was  not 
appropriate.  Neither  is  a direct  comparison  to  the  results  of  the  1990  main  science  assessment.  The  1996 
main  science  assessment  can  be  used  to  start  a new  baseline  for  measuring  trends  in  the  nation. 

The  pool  of  items  used  in  the  1996  science  assessment  contained  a range  of  constructed-response 
and  multiple-choice  questions  measuring  performance  on  sets  of  objectives.  The  items  in  the  assessment 
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were  based  on  the  curriculum  framework  described  in  Science  Framework  for  the  1996  National 
Assessment  of  Educational  Progress  (National  Assessment  Governing  Board,  1993),  The  total  number  of 
scaled  items  was  136,  190,  and  186,  respectively,  for  grades  4,  8,  and  12.  Note  that  some  items  overlap 
across  grade.  Each  of  the  items  was  classified  into  one  of  three  fields  of  science:  earth  science;  physical 
science;  and  life  science.  These  three  fields  of  science  constituted  the  scales  used  in  1996  reporting.  Table 
13-2  shows  the  numbers  of  items  within  content  area  scales  for  each  grade.  The  numbers  presented  in 
Table  13-2  show  item  counts  both  for  the  original  item  pool,  and  after  the  necessary  adjustments  were 
made  during  scaling  (see  Section  13.5.2,  below). 


Table  13-2 

Number  of  Items  in  Scales  in  the  Science  Main  Assessment  by  Field  of  Science 


Grade 

Physical 

Science 

Earth 

Science 

Life 

Science 

Total 

4 

Pre-Scaling 

45 

53 

47 

145 

Post-Scaling 

43 

49 

44 

136 

8 

Pre-Scaling 

63 

65 

66 

194 

Post-Scaling 

62 

63 

65 

190 

12 

Pre-Scaling 

60 

64 

66 

190 

Post-Scaling 

59 

62 

65 

186 

For  each  grade,  the  items  were  divided  into  15  mutually  exclusive,  separately  timed  blocks.  At 
grade  4,  students  were  given  20  minutes  to  complete  each  block;  at  grades  8 and  12  each  block  required 
30  minutes.  As  described  in  Chapter  2,  the  blocks  were  combined  into  booklets  according  to  a complex 
spiraling  design.  (See  Chapter  4 for  more  information  about  the  blocks  and  booklets.)  Each  student’s 
booklet  contained  three  blocks  of  cognitive  items.  Four  of  the  15  blocks  were  hands-on  tasks  in  which 
students  were  given  a set  of  equipment  and  asked  to  conduct  an  investigation  and  answer  questions 
(mostly  constructed-response)  relating  to  the  investigation.  These  hands-on  tasks  were  always  presented 
in  the  last  position,  after  two  paper-and-pencil  blocks.  Three  of  the  remaining  1 1 blocks  were  theme 
blocks.  Theme  blocks  were  placed  randomly  in  student  booklets,  but  not  in  every  booklet.  No  student 
received  more  than  one  theme  block.  Each  theme  block  was  paired  with  each  non-theme  paper-and-pencil 
block  just  once.  Each  paper-and-pencil  block  appeared  in  the  first  or  second  position  the  same  number  (3 
or  4)  of  times.  For  each  of  the  grades,  the  composition  of  each  block  of  items,  in  terms  of  content  and 
format,  is  given  in  Tables  13-3  through  13-5.^  Common  labeling  of  these  blocks  across  grade  levels  does 
not  denote  common  items. 


^ The  numbers  in  Tables  13-2  through  13-8  differ  slightly  from  those  given  in  Chapter  2.  The  numbers  in  Chapter  2 do  not  reflect  the 
grouping  of  certain  sets  of  items  into  cluster  items  for  the  purposes  of  scaling. 
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Table  13-3 

1 996  NAEP.  Science  Block  Composition  by  Item  Type,  Grade  4,  As  Defined  Before  Scaling 


Block 

Multiple- 
Choice  Items 

Constructed-Response  Items  Scored  Polytomously 
2-  3-  4-  5- 

category  category  category  category 

Cluster 

Items 

Total 

Items 

S3 

0 

1 

6 

0 

0 

0 

7 

S4 

1 

4 

1 

1 

0 

0 

7 

S5 

2 

0 

4 

1 

1 

0 

8 

S6 

0 

0 

3 

2 

0 

0 

5 

S7 

2 

0 

7 

1 

0 

0 

10 

S8 

1 

0 

7 

0 

1 

0 

9 

S9 

2 

0 

6 

1 

0 

0 

9 

SIO 

6 

0 

6 

0 

0 

0 

12 

sn 

6 

1 

4 

0 

0 

0 

11 

S12 

6 

0 

8 

1 

0 

0 

15 

S13 

6 

0 

4 

1 

0 

0 

11 

S14 

5 

0 

5 

0 

0 

0 

10 

S15 

3 

0 

5 

1 

0 

0 

9 

S20 

6 

0 

2 

3 

0 

0 

11 

S21 

5 

0 

4 

1 

1 

0 

11 

Total 

51 

6 

72 

13 

3 

0 

145 

Table  13-4 

1996  NAEP  Science  Block  Composition  by  Item  Type,  Grade  8,  As  Defined  Before  Scaling 

Multiple- 

Constructed-Response  Items  Scored  Polytomously 
2-  3-  4-  5- 

Cluster 

Total 

Block 

Choice  Items 

category 

category 

category 

category 

Items 

Items 

S3 

0 

0 

4 

1 

0 

1 

6 

S4 

3 

0 

4 

3 

0 

0 

10 

S5 

0 

0 

8 

0 

0 

0 

8 

S6 

0 

0 

5 

2 

0 

0 

7 

S7 

2 

2 

8 

0 

0 

0 

12 

S8 

5 

0 

5 

0 

0 

0 

10 

S9 

3 

0 

9 

1 

0 

0 

13 

SIO 

8 

1 

6 

1 

0 

0 

16 

Sll 

8 

0 

7 

1 

0 

0 

16 

S12 

8 

1 

5 

2 

0 

0 

16 

S13 

8 

0 

7 

1 

0 

0 

16 

S14 

7 

0 

8 

0 

0 

1 

16 

S15 

7 

1 

6 

1 

1 

0 

16 

S20 

8 

0 

6 

2 

.0 

0 

16 

S21 

7 

0 

7 

2 

0 

0 

16 

Total 

74 

5 

95 

17 

1 

2 

194 
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Table  13-5 

1996  NAEP  Science  Block  Composition  by  Item  Type,  Grade  12,  As  Defined  Before  Scaling 


Block 

Multiple- 
Choice  Items 

Constructed-Response  Items  Scored  Polytomously 
2-  3-  4-  5- 

category  category  category  category 

Cluster 

Items 

Total 

Items 

S3 

0 

0 

4 

1 

1 

0 

6 

S4 

0 

0 

1 

1 

2 

0 

4 

S5 

0 

1 

7 

0 

0 

0 

8 

S6 

0 

1 

5 

1 

1 

0 

8 

S7 

5 

0 

7 

3 

0 

0 

15 

S8 

6 

0 

7 

0 

0 

0 

13 

S9 

4 

0 

8 

1 

1 

0 

14 

SIO 

8 

0 

9 

1 

0 

0 

18 

Sll 

8 

1 

4 

3 

0 

0 

16 

S12 

8 

1 

5 

2 

0 

0 

16 

S13 

8 

0 

7 

1 

0 

0 

16 

S14 

8 

0 

6 

1 

1 

0 

16 

S15 

0 

1 

5 

2 

0 

0 

8 

S20 

7 

0 

6 

3 

0 

0 

16 

S21 

8 

0 

4 

4 

0 

0 

16 

Total 

70 

5 

85 

24 

6 

0 

190 

Some  items  (fewer  than  10%)  received  special  treatment  during  scaling.  For  each  of  the  grades, 
Tables  13-6  through  13-8  show  the  composition  of  each  block  after  deletions  of  items  and  collapsing  of 
categories  for  polytomously-scored  constructed-response  items  as  a result  of  scaling.  If  data  had  poor  fit 
with  the  response  model  for  an  item,  the  item  was  deleted.  If  a constructed-response  item  was  scored  in 
multiple  categories  but  one  category  had  no  (or  very  few)  responses,  or  one  of  the  categories  had 
responses  that  had  poor  fit  to  the  response  model,  that  category  was  combined  with  other  categories 
(“collapsed”).  All  item  deletions  and  all  but  one  category  collapse  were  performed  in  the  course  of  scaling 
the  national  science  assessment  data;  the  remaining  collapse  was  performed  based  on  data  in  State 
Assessment,  with  the  same  collapse  performed  for  the  national  scaling.  In  addition,  categories  of  a small 
number  of  items  were  combined  These  changes  were  made  so  that  the  scaling  model  used  for  these  items 
fit  the  data  more  closely,  and  are  described  more  fully  in  Section  13.5. 

For  grade  4,  each  of  the  1 1 paper-and-pencil  blocks  contained  from  five  to  nine  constructed- 
response  items.  Seven  of  these  blocks  contained  one  or  more  constructed-response  items  scored  on  a 0-3 
scale.  Two  items  were  scored  on  a 0-4  scale.  The  four  hands-on  task  blocks  contained  from  five  to  seven 
constructed-response  items  and  up  to  two  multiple-choice  items. 

For  grade  8,  each  of  the  1 1 paper-and-pencil  blocks  contained  from  five  to  ten  constructed- 
response  items.  Eight  of  these  blocks  contained  one  or  more  constructed-response  items  scored  on  a 0-3 
scale.  One  item  was  scored  on  a 0-4  scale.  The  four  hands-on  task  blocks  contained  from  six  to  eight 
constructed-response  items.  One  of  these  blocks  also  contained  three  multiple-choice  items. 

For  grade  12,  each  of  the  11  paper-and-pencil  blocks  contained  from  seven  to  ten  constructed- 
response  items.  Ten  of  these  blocks  contained  one  or  more  constructed-response  items  scored  on  a 0-3 
scale.  Two  items  were  scored  on  a 0-4  scale.  The  four  hands-on  task  blocks  contained  from  four  to  eight 
constructed-response  items.  None  of  these  blocks  contained  multiple-choice  items. 
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Table  13-6 

1996  NAEP  Science  Block  Composition  by  Item  Type,  Grade  4,  As  Defined  After  Scaling* 


Block 

Multiple- 
Choice  Items 

Constructed-Response  Items  Scored  Polytomously 
2-  3-  4-  5- 

category  category  category  category 

Cluster 

Items 

Total 

Items 

S3 

0 

1 

4 

0 

0 

1 

6 

S4 

1 

5 

1 

0 

0 

0 

7 

S5 

2 

0 

4 

0 

1 

0 

7 

S6 

0 

0 

3 

1 

0 

0 

4 

S7 

2 

3 

4 

1 

0 

0 

10 

S8 

1 

0 

7 

0 

1 

0 

9 

S9 

1 

0 

7 

0 

0 

0 

8 

SIO 

6 

0 

4 

0 

0 

1 

11 

Sll 

6 

2 

3 

0 

0 

0 

11 

S12 

6 

0 

5 

0 

0 

0 

11 

S13 

6 

1 

3 

1 

0 

0 

11 

S14 

5 

0 

. 5 

0 

0 

0 

10 

S15 

3 

0 

5 

1 

0 

0 

9 

S20 

6 

0 

2 

3 

0 

0 

11 

S21 

5 

1 

3 

1 

1 

0 

11 

Total 

50 

13 

60 

8 

3 

2 

136 

*Counts  reflect  items  that  were 

dropped  and  collapsed. 

Table  13-7 

1996  NAEP  Science  Block  Composition  by  Item  Type,  Grade  8,  As  Defined  After  Scaling* 

Constructed-Response  Items  Scored  Polytomously 

Multiple- 

2- 

3- 

4- 

5- 

Cluster 

Total 

Block 

Choice  Items 

category 

category 

category 

category 

Items 

Items 

S3 

0 

1 

3 

1 

0 

1 

6 

S4 

2 

1 

4 

1 

0 

1 

9 

S5 

0 

1 

7 

0 

0 

0 

8 

S6 

0 

1 

.4 

1 

0 

0 

6 

S7 

2 

4 

6 

0 

0 

0 

12 

S8 

5 

1 

4 

0 

0 

0 

10 

S9 

3 

2 

7 

1 

0 

0 

13 

SIO 

8 

1 

7 

0 

0 

0 

16 

Sll 

8 

1 

6 

1 

0 

0 

16 

S12  • 

8 

1 

5 

2 

0 

0 

16 

S13 

8 

1 

4 

1 

0 

1 

15 

S14 

7 

0 

8 

0 

0 

1 

16 

S15 

6 

1 

6 

1 

1 

0 

15 

S20 

8 

2 

4 

2 

0 

0 

16 

S21 

7 

0 

7 

2 

0 

0 

16 

Total 

72 

18 

82 

13 

1 

4 

190 

* Counts  reflect  items  that  were  dropped  and  collapsed. 


Table  13-8 

1996  NAEP  Science  Block  Composition  by  Item  Type,  Grade  12,  As  Defined  After  Scaling* 


Multiple- 

Block  Choice  Items 

Constructed-Response  Items  Scored  Polytomously 
2-  3-  4-  5- 

category  category  category  category 

Cluster 

Items 

Total 

Items 

S3 

0 

1 

4 

1 

0 

0 

6 

S4. 

0 

0 

2 

2 

0 

0 

4 

S5 

0 

2 

6 

0 

0 

0 

8 

S6 

0 

4 

2 

1 

1 

0 

8 

S7 

5 

3 

5 

2 

0 

0 

15 

S8 

6 

2 

4 

0 

0 

0 

12 

S9 

4 

: 3 

5 

1 

1 

0 

14 

SIO 

8 

0 

9 

1 

0 

0 

18 

Sll 

8 

1 

4 

3 

0 

0 

16 

S12 

8 

1 

5 

2 

0 

0 

16 

S13 

8 

0 

5 

1 

0 

1 

15 

S14 

8 

0 

6 

2 

0 

0 

16 

S15 

0 

2 

4 

2 

0 

0 

8 

S20 

6 

0 

6 

2 

0 

0 

14 

S21 

8 

0 

4 

4 

0 

0 

16 

Total 

69 

19 

71 

24 

2 , 

1 

186 

* Counts  reflect  items  that  were  dropped  and  collapsed. 


All  constructed-response  items  were  scored  by  specially  trained  readers,  as  described  in  Chapter 
5.  In  addition,  a small  number  of  “cluster  items”  were  formed.  A cluster  item  is  an  aggregation  of  a group 
of  items  (in  the  case  of  NAEP  science,  typically  two  to  four  items)  that  are  related  to  a single  content  strand, 
topic,  or  stimulus,  and  are  developed  and  scored  as  a single  unit  (see  Wainer  & Kiely,  1987,  for  further 
details  and  examples  of  different  types  of  cluster  items).  Some  items  were  initially  scored  as  cluster  items, 
and  the  additional  clusters  were  formed  in  scaling  due  to  data  dependencies. 

In  the  main  samples,  each  student  was  administered  a booklet  containing  two  paper-and-pencil 
blocks  and  one  block  consisting  of  a hands-on  task.  In  addition,  the  booklet  contained  a block  of 
background  questions  common  to  all  booklets  for  a particular  grade  level,  a block  of  questions  concerning 
the  student’s  motivation  and  his  or  her  perception  of  the  difficulty  of  the  cognitive  items,  and  a block  of 
science-related  background  questions  common  to  all  science  booklets  for  a particular  grade  level. 

The  design  of  the  1996  science  assessment  required  that  each  student  be  administered  one  of  the 
37  booklets  in  the  design.  Within  each  administration  site,  all  booklets  were  “spiraled”  together  in  a 
random  sequence  and  distributed  to  students  sequentially,  in  the  order  of  the  students’  names  on  the 
Student  Listing  Form  (see  Chapter  4).  As  a result  of  the  design  and  the  spiraling  of  booklets,  a 
considerable  degree  of  balance  was  achieved  in  the  data  collection  process.  Each  block  of  items  (and, 
therefore,  each  item)  was  administered  to  randomly  equivalent  samples  of  students. 


13.3  SPECIAL  SCIENCE  ASSESSMENT 

As  stated  previously,  there  was  a special  study  in  the  1996  national  NAEP  assessment  in 
addition  to  the  main  and  long-term  trend  assessments.  This  was  the  advanced  study,  denoted  by 
the  12[Sci-Advanced]  sample  in  Table  13-1.  This  study  examined  the  performance  of  twelfth- 
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grade  students  who  were  taking  advanced  science  courses.  Students  were  assessed  for 
approximately  two  hours,  and  each  student  received  four  cognitive  blocks,  consisting  of  a 
common  block  (SS,  composed  of  18  items)  and  three  special  blocks  (each  composed  of  16  items) 
designed  to  assess  advanced  material.  Each  block  assessed  specific  science  content:  one  block  for 
physics,  one  for  chemistry,  and  one  for  life  sciences.  The  common  block  was  composed  of  items 
from  the  main  assessment,  although  these  items  were  drawn  from  several  different  blocks  from 
the  main  assessment.  The  block  structure  of  the  special  study  booklets  is  provided  in  Table  13-9. 
In  addition  to  the  cognitive  blocks,  the  special  study  booklets  had  three  blocks  in  common  with 
the  main  assessment: 


1 . a general  student  background  block  (CS) 

2.  a science  background  block  (SB),  and 

3.  a motivation  block  (SX). 


Table  13-9 

Block  Structure  of  the  12[Sci-Advanced]  Special  Study  Booklets 


Booklet 

IDs 

First 

Cognitive  Blocks 
Second  Third 

Fourth 

238 

SS 

SR 

SV 

SP 

239 

SS 

SV 

SP 

SR 

240 

SS 

SP 

SR 

SV 

The  advanced  study  was  not  part  of  the  main  assessment  and  analyses  for  these  booklets 
will  be  described  in  an  NCES  publication  by  Christine  O’Sullivan  to  be  published  in  the  third 
quarter  of  1999.  Therefore,  the  special  12[Sci-Advanced]  sample  will  not  be  discussed  further  in 
this  chapter. 


13.4  ITEM  ANALYSES 

13.4.1  Conventional  Item  and  Test  Analyses 

This  section  contains  a detailed  description  of  the  conventional  item  analysis  performed 
on  the  science  data.  This  analysis  was  done  within  block  so  that  a student’s  score  is  the  sum  of 
item  scores  in  a block.  Dichotomous  items  (multiple-choice  and  2-category  constructed-response) 
were  scored  as  right  or  wrong.  Polytomous  items  were  not  scored  right/wrong  but  were  scored 
with  three  or  more  categories  reflecting  several  degrees  of  knowledge. 

Tables  13-10,  13-11,  and  13-12  show  the  number  of  items,  average  weighted  item  score, 
average  weighted  item-to-total  score  correlation  (biserial  or  polyserial),  and  weighted  alpha 
reliability  for  each  block  administered  at  each  grade  level  for  the  main  assessment.  The  table  also 
gives  the  number  of  students  who  were  administered  the  block  and  the  weighted  percent  reaching 
the  last  item  in  the  block.  Student  sampling  weights  were  used  to  compute  all  statistics  except  for 
the  sample  sizes.  Preliminary  item  analyses  for  all  items  within  a block  were  completed  before 
scaling;  however,  the  results  shown  here  reflect  the  characteristics  of  the  items  that  contributed  to 
the  final  science  scales. 
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Table  13-10 

Descriptive  Statistics  for  Item  Blocks  by  Position  Within  Test  Booklet  and  Overall 
Occurrences  for  the  Science  Main  Sample,  Grade  4,  As  Defined  After  Scaling 

Statistic  Position  S7  S8  S9  SIO  Sll  S12  S13  S14  S15  S20  S21 
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Table  13-11 

Descriptive  Statistics  for  Item  Blocks  by  Position  Within  Test  Booklet  and  Overall 
Occurrences  for  the  Science  Main  Sample,  Grade  8,  As  Defined  After  Scaling 

Statistic  Position  S7  S8  S9  SIO  Sll  S12  S13  S14  SIS  S20  S21 
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Table  13-12 

Descriptive  Statistics  for  Item  Blocks  by  Position  Within  Test  Booklet  and  Overall 
Occurrences  for  the  Science  Main  Sample,  Grade  12,  As  Defined  After  Scaling 

Statistic Position  S7 S8 S9 SIO  Sll  S12  S13 S14 S15 S20 _ 
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As  described  in  Chapter  9,  in  NAEP  analyses  (both  conventional  and  IRT-based),  a distinction 
is  made  between  missing  responses  at  the  end  of  each  block  (not  reached)  and  missing  responses  prior 
to  the  last  observed  response  (omitted).  Standard  practice  at  ETS  is  to  treat  all  nonrespondents  to  the 
last  item  as  if  they  had  not  reached  the  item.  Items  that  were  not  reached  were  treated  as  if  they  had 
not  been  presented  to  the  examinee,  while  omitted  items  were  regarded  as  incorrect.  The  proportion  of 
students  attempting  the  last  item  of  a block  (or,  equivalently,  one  minus  the  proportion  not  reaching 
the  last  item)  can  be  used  as  an  index  of  the  degree  of  speededness  of  the  block  of  items. 

As  is  evident  from  Tables  13-10  through  13-12,  the  difficulty  and  the  average  item-to-total  correlations 
of  the  blocks  varied  somewhat.  Such  variability  was  expected  since  these  blocks  were  not  created  to  be  parallel  in 
either  difficulty  or  content.  Based  on  the  proportion  of  students  attempting  the  last  item,  no  block  seemed  to  be 
speeded,  by  the  criterion  of  a proportion  less  than  .8  attempting  the  last  item.  The  most  speeded  block  showed  84 
percent  of  the  students  reaching  the  last  item  in  the  block. 

For  the  1 1 paper-and-pencil  blocks,  small  but  consistent  differences  were  noted  based  upon 
whether  a block  occurred  in  the  first  or  second  position  within  a booklet.  When  the  block  appeared  first 
in  the  booklet,  the  average  item  score  tended  to  be  higher  and  the  average  polyserial  correlation  tended 
to  be  lower.  The  largest  differences  were  noted  in  the  proportion  of  students  not  attempting  the  last  item 
in  the  block;  more  students  attempted  the  last  item  when  the  block  appeared  in  the  second  position.  It 
appears  that  the  students  learned  to  pace  themselves  through  the  second  block,  based  on  their  experience 
with  the  first  block.  Similar  effects  (slightly  larger)  were  noted  in  the  1992  NAEP  reading  assessment 
(Donoghue,  1994).  At  that  time,  a study  was  completed  to  examine  the  effect  of  the  serial  position 
differences.  Due  to  the  balance  of  the  complex  design  of  the  booklets,  the  serial  position  differences 
were  found  to  have  minimal  effects  on  scaling. 

13.4.2  Scoring  the  Constructed-Response  Items 

As  indicated  in  Table  13-4  through  13-6,  about  two-thirds  of  the  science  items  were 
constructed-response.  Two-category  constructed-response  items  were  given  a right/wrong  scoring. 

The  categories  of  responses  for  the  items  and  the  number  of  responses  that  were  rescored  for  each 
item  are  indicated  in  Appendix  I.  The  percent  agreement  for  the  raters  and  Cohen’s  (1968)  Kappa,  a 
reliability  estimate  appropriate  for  items  that  are  dichotomized,  are  also  given  in  the  tables.  For  grades 
4 and  12,  a 20  percent  sample  was  used  in  calculating  the  reliability.  At  grade  8,  the  national  and  State 
Assessment  data  were  combined.  A 6 percent  sample  of  these  combined  data  was  used  to  calculate  the 
reliability  results. 

; 

In  general,  the  rater  reliability  of  the  scoring  for  dichotomized  responses  was  quite  high. 

Cohen’s  Kappa  reliabilities  ranged  over  items  from  .86  to  .93  for  grade  4,  from  .75  to  .96  for  grade  8, 
and  from  .71  to  .95  for  grade  12. 

Chapter  7 discusses  the  definition  of  the  item  ratings  and  describes  the  process  by  which  teams  of 
raters  scored  the  constructed-response  items.  This  discussion  includes  the  rating  definitions  for  regular,  short 
and  extended  constructed-response  items  as  well  as  the  range  of  interrater  reliabilities  that  were  obtained. 
Extended  constructed-response  items  were  scored  on  a scale  from  1 to  5 to  reflect  degrees  of  knowledge.  In 
scaling,  this  scale  is  shifted  to  0 to  4.  Rating  information  on  extended  constructed-response  items  can  be  found 
in  Appendix  I,  which  lists  the  sample  sizes,  percent  agreement,  and  Cohen’s  Kappa  reliability  index. 

13.4.3  Differential  Item  Functioning 

A differential  item  functioning  (DIF)  analysis  of  the  science  items  was  done  to  identify  potentially 
biased  items.  Sample  sizes  were  large  enough  to  compare  male  and  female  students.  White  and  Black  students, 
and  White  and  Hispanic  students.  The  purpose  of  the  analysis  was  to  identify  items  that  should  be  examined 
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more  closely  by  a committee  of  trained  test  developers  and  subject-matter  specialists  for  possible  bias  and 
consequent  exclusion  from  the  assessment.  The  presence  of  DIF  in  an  item  means  that  the  item  is  differentially 
harder  for  one  group  of  students  than  another,  while  controlling  for  the  ability  level  of  the  students. 

For  dichotomous  items,  the  Mantel-Haenszel  procedure  as  adapted  by  Holland  and  Thayer  (1988)  was 
used  as  a test  of  DIF  (this  is  described  in  Chapter  9).  The  Mantel  procedure  (Mantel,  1963)  was  used  for 
detection  of  DIF  in  polytomous  items  and  also  as  described  by  Zwick,  Donoghue,  and  Grima  (1993).  This 
procedure  assumes  ordered  categories. 

For  DIF  analyses*  weights  were  rescaled  separately  for  each  comparison,  as  described  in  Chapter  9. 
DIF  analyses  were  conducted  separately  by  grade.  For  dichotomous  items,  the  DIF  index  generated  by  the 
Mantel-Haenszel  procedure  was  used  to  place  items  into  one  of  three  categories:  “A,”  “B,”  or  C.  A items 
exhibit  no  DIF,  “B”  items  exhibit  a weak  indication  of  DIF,  and  “C”  items  exhibit  a strong  indication  of  DIF. 
“C”  items  were  examined  by  a DIF  committee  for  presence  of  bias. 

Table  13-13  summarizes  the  results  of  the  DIF  analyses  for  dichotomous  items.  Focal  groups  are 
female.  Black,  and  Hispanic  groups. 


Table  13-13 

DIF  Category  by  Grade  for  Dichotomous  Items 


Grade 

DIF 

Category* 

Male/Female 

Analysis 

White/Black 

White/Hispanic 

4 

C- 

3 

0 

0 

B- 

13 

4 

1 

A- 

21 

26 

33 

A-i- 

19 

26 

23 

B-i- 

2 

2 

1 

C-i- 

0 

0 

0 

8 

c- 

3 

0 

0 

B- 

10 

6 

2 

A- 

41 

33 

34 

A-i- 

23 

40 

43 

B-i- 

4 

2 

2 

C-i- 

0 

0 

0 

12 

c- 

1 

0 

0 

B- 

13 

4 

2 

A- 

38 

31 

33 

A-i- 

22 

35 

41 

B-i- 

5 

9 

3 

C-1- 

0 

0 

0 

‘ Positive  values  of  the  index  indicate  items  that  are  differentially  easier  for  the  focal  group  (female, 
Black,  or  Hispanic  students)  than  for  the  reference  groups  (male  or  White  students).  “A+”  or  “A-”  ^ 
means  no  indication  of  DIF,  “B+”  means  a weak  indication  of  DIF  in  favor  of  the  focal  group,  “B-“ 
means  a weak  indication  of  DIF  in  favor  of  the  reference  group  and  “C+”  or  “C-”  means  a strong 
indication  of  DIF. 


Positive  values  indicate  items  that  were  differentially  easier  for  the  focal  group.  Table  13-14  summarizes  the 
results  of  the  DIF  analyses  for  polytomous  items.  Again,  focal  groups  are  female.  Black,  and  Hispanic  groups, 
and  positive  values  indicate  that  the  item  was  differentially  easier  for  the  focal  group.  The  Mantel  statistic 
provides  a statistical  test  of  the  hypothesis  of  no  DIF.  To  aid  in  interpreting  the  results  for  polytomous  items, 
the  standardized  mean  difference  between  focal  and  reference  groups  was  produced.  This  statistic  was  rescaled 
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by  dividing  the  standardized  mean  differences  by  the  standard  deviation  of  the  respective  item.  The  description 
of  this  procedure  can  be  found  in  Chapter  9.  For  polytomous  items,  a standardized  mean  difference  ratio  of  .25 
or  greater  (coupled  with  a significant  Mantel  statistic)  was  considered  a strong  indication  of  DIF.  It  can  be 
shown  that  standardized  mean  difference  ratios  of  .25  are  at  least  as  extreme  as  Mantel-Haenszel  statistics 
corresponding  to  “C”  items  (see  Chapter  9 for  details). 


Table  13-14 

DIF  Category  by  Grade  for  Polytomous  Items 


Grade 

DIF 

Category* 

Male/Female 

Analysis 

White/Black 

White/Hispanic 

4 

CC- 

5 

2 

0 

BB- 

4 

2 

1 

AA- 

39 

39 

40 

AA+ 

29 

36 

41 

BB+ 

4 

2 

1 

CC+ 

2 

2 

0 

8 

CC- 

7 

0 

2 

BB- 

7 

5 

3 

AA- 

43 

48 

54 

AA+ 

57 

57 

54 

BB+ 

3 

4 

3 

CC+ 

1 

4 

2 

12 

CC- 

3 

3 

2 

BB- 

7 

2 

2 

AA- 

40 

54 

54 

AA+ 

49 

45 

45 

BB+ 

9 

5 

4 

CC+ 

3 

2 

4 

' Positive  values  of  the  index  indicate  items  that  are  differentially  easier  for  the  focal  group  (female, 
Black,  or  Hispanic  students)  than  for  the  reference  groups  (male  or  White  students).  “A+”  or  “A-” 
means  no  indication  of  DIF,  “B+”  means  a weak  indication  of  DIF  in  favor  of  the  focal  group,  “B-“ 
means  a weak  indication  of  DIF  in  favor  of  the  reference  group  and  “C+”  or  “C-”  means  a strong 
indication  of  DIF. 


Following  standard  practice  at  ETS,  all  items  identified  as  showing  DIF  were  reviewed  by  a 
committee  of  trained  test  developers  and  subject-matter  specialists.  As  described  in  Chapter  9,  such 
committees  are  charged  with  making  judgments  about  whether  the  differential  difficulty  of  an  item  is  unfairly 
related  to  group  membership;  that  is,  whether  the  item  is  biased.  The  committee  assembled  to  review  NAEP 
items  that  were  identified  as  “C”  or  “CC”  items.  The  committee  included  both  ETS  staff  and  outside  members 
with  expertise  in  the  field.  It  was  the  committee’s  judgment  that  one  of  the  items  for  the  national  data  was 
functioning  differentially  due  to  factors  irrelevant  to  test  objectives.  The  item  asked  the  student  to  list  two 
ways  that  cold  temperatures  could  cause  problems.  Although  the  item  appeared  to  be  disadvantaging  Hispanic 
students,  the  committee  concluded  that  this  was  probably  because  a large  proportion  of  Hispanic  Americans 
live  in  warmer  parts  of  the  country,  and  that  anyone  without  experience  of  cold  weather  would  be 
disadvantaged  in  answering  this  question.  The  item  was  removed  from  scaling  due  to  this  differential  item 
functioning. 
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13.5  ITEM  RESPONSE  THEORY  (IRT)  SCALING 


In  1993,  the  National  Assessment  Governing  Board  (NAGB)  determined  that  future  NAEP 
assessments  should  be  developed  using  within-grade  frameworks.  Within-grade  scaling  removes  the  constraint 
that  the  trait  being  measured  is  cumulative  across  the  grade  levels  of  the  assessment.  It  also  means  that  there  is 
no  need  for  overlap  items  across  grades.  Consistent  with  this  view,  NAGB  also  declared  that  scaling  be 
performed  within-grade.  Any  items  that  happened  to  be  the  same  across  grades  in  the  assessment  were  scaled 
separately  for  each  grade,  thus  making  it  possible  for  common  items  to  function  differently  in  the  separate 
grades.  Therefore,  the  Science  Framework  for  the  1996  National  Assessment  of  Educational  Progress 
(National  Assessment  Governing  Board,  1993)  specifies  that  the  1996  science  assessment  be  developed 
within-grade.  Likewise,  all  IRT  scaling  was  performed  within-grade. 

Within  each  grade,  items  were  grouped  into  three  distinct  sets  corresponding  to  the  three  fields  of 
science:  earth,  physical,  and  life.  IRT-based  scales  corresponding  to  each  of  the  fields  of  science  above  were 
developed  using  the  scaling  models  described  in  Chapter  11.  The  scales  summarize  student  performance 
across  all  three  item  types  in  the  assessment  (multiple-choice,  short  constructed-response,  and  extended 
constructed-response). 


13.5.1  Item  Parameter  Estimation 

For  each  fields  of  science  scale,  item  parameter  estimates  were  obtained  by  the  NAEP 
BILOG/PARSCALE  program,  which  combines  Mislevy  and  Bock’s  (1982)  BILOG  and  Muraki  and  Bock’s 
(1991)  PARSCALE  computer  programs.  The  program  uses  marginal  estimation  procedures  to  estimate  the 
parameters  of  the  one-,  two-,  and  three-parameter  logistic  models,  and  the  generalized  partial  credit  model 
described  by  Muraki  (1992)  (see  Chapter  1 1).  The  calibration  was  performed  using  all  the  available  examinees 
in  the  reporting  group.  Student  sampling  weights  were  used  for  the  analysis. 

As  described  in  Chapter  1 1,  multiple-choice  items  were  dichotomously  scored  (scored  0,1)  and  were 
scaled  using  the  three-parameter  logistic  model.  Omitted  responses  to  multiple-choice  items  were  treated  as 
fractionally  correct,  with  the  fraction  being  set  to  the  reciprocal  of  the  number  of  response  options  for  an  item. 
All  constructed-response  items  with  two  categories  were  dichotomously  scored  and  were  scaled  using  the  two- 
parameter  logistic  model  with  the  lower  asymptote  parameter  set  at  6.  Omitted  responses  to  these  items  were 
treated  as  incorrect.  For  calibration,  all  items  that  were  not  reached  were  treated  as  if  they  were  not  presented 
to  the  examinees.  Responses  to  extended  constructed-response  items  that  were  off-task  were  also  treated  as  if 
they  had  not  been  presented. 

A key  assumption  associated  with  IRT  scales  is  that  of  conditional  independence.  Conditional  on 
proficiency, .examinee’s  item  responses  are  assumed  to  be  independent.  When  sets  of  items  are  logically 
dependent  on  each  other,  or  are  based  on  a single  stimulus,  this  assumption  can  be  violated  to  a degree  that 
results  in  aberrant  scaling  results.  In  order  to  avoid  possible  problems  with  inter-item  dependencies,  a small 
number  of  additional  cluster  items  was  created  by  combining  examinee  responses  to  sets  of  related  items  into  a 
single  score  for  each  set.  The  cluster  items,  rather  than  their  original  constituent  items,  were  used  in  scaling  the 
1996  science  assessment.  Examinees  omitting  all  constituents  of  the  cluster  item  were  placed  in  the  “zero 
correct”  category  of  the  cluster  item.  Examinees  classified  as  “not  reaching”  all  constituent  parts  were  treated 
as  having  not  been  presented  the  cluster  item.  All  cluster  items  were  scaled  using  the  generalized  partial  credit 
model. 

Each  of  the  multi-category  constructed  response  items  was  also  scaled  using  the  generalized  partial 
credit  model.  These  items  had  two,  three,  four,  or  five  categories  of  partial  credit.  One  cluster  item  had  six 
categories.  Scoring  levels  were  labeled  as  shown  in  Table  13-15. 
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Table  13-15 


Labels  for  Score  Levels  of  Polytomous  Items 


Score 

3-Category 

4-Category 

5-Category 

6-Category 

5 

Complete 

4 

Complete 

Essential 

3 

Complete 

Essential 

Adequate 

2 

Complete 

Partially  correct 

Partially  correct 

Partially  correct 

1 

Partially  correct 

Unsatisfactory 

Unsatisfactory 

Unsatisfactory 

0 

Wrong,  off-task,  or 

Wrong,  off-task,  or 

Wrong,  off-task,  or 

Wrong,  off-task,  or 

omitted 

omitted 

omitted 

omitted 

Empirical  Bayes  modal  estimates  of  all  item  parameters  were  obtained  from  the  BDLOG/PARSCALE 
program.  Prior  distributions  were  imposed  on  item  parameters  with  the  following  starting  values:  thresholds 
(normal  [0,2]),  slopes  (log-normal  [0,.5]);  and  asymptotes  (two-parameter  beta  with  parameter  values 
determined  as  functions  of  the  number  of  response  options  for  an  item  and  a weight  factor  of  50).  The 
locations  (but  not  the  dispersions)  of  the  item  parameter  prior  distributions  were  updated  at  each  program 
estimation  cycle  in  accordance  with  provisional  estimates  of  the  item  parameters. 

Starting  values  were  computed  from  item  statistics.  Item  parameter  estimation  proceeded  in  two 
phases.  First,  the  subject  ability  distribution  was  assumed  fixed  (normal  [0,1])  and  a stable  solution  was 
obtained.  The  parameter  estimates  from  this  solution  were  then  used  as  starting  values  for  a subsequent  set  of 
runs  in  which  the  subject  ability  distribution  was  freed  and  estimated  concurrently  with  item  parameter 
estimates.  After  each  estimation  cycle,  the  subject  ability  distribution  was  restandardized  to  have  a mean  of 
zero  and  standard  deviation  of  one.  Correspondingly,  parameter  estimates  for  that  cycle  were  also  linearly 
restandardized. 


13.5.2  Evaluation  of  Model  Fit 

During  and  subsequent  to  item  parameter  estimation,  an  evaluation  of  the  fit  of  the  IRT  models  was 
carried  out  for  each  of  the  items  in  the  item  pool.  These  evaluations  were  conducted  to  determine  the  final 
composition  of  the  item  pool  making  up  the  scales  by  identifying  misfitting  items  that  should  not  be  included. 
Evaluations  of  model  fit  were  based  primarily  on  graphical  analyses.  For  dichotomously-scored  multiple- 
choice  and  two-category  response  items,  model  fit  was  evaluated  by  examining  plots  of  estimates  of  the 
expected  conditional  (on  9)  probability  of  a correct  response  that  do  not  assume  a two-parameter  or  three- 
parameter  logistic  model  versus  the  probability  predicted  by  the  estimated  item  characteristic  curve  (see 
Mislevy  & Sheehan,  1987,  p.  302).  For  the  cluster  items  and  multiple-category  constructed-response  items, 
similar  plots  were  produced  for  each  item  category  characteristic  curve  (see  Chapter  9). 

As  with  most  procedures  that  involve  evaluating  plots  of  data  versus  model  predictions,  a certain 
degree  of  subjectivity  is  involved  in  determining  the  degree  of  fit  necessary  to  justify  use  of  the  model.  There 
are  a number  of  reasons  why  evaluation  of  model  fit  relied  primarily  on  analyses  of  plots  rather  than  seemingly 
more  objective  procedures  based  on  goodness-of-fit  indices  such  as  the  “pseudo  chi-squares”  produced  in 
BILOG  (Mislevy  & Bock,  1982).  First,  the  exact  sampling  distributions  of  these  indices  when  the  model  fits 
are  not  well  understood,  even  for  fairly  long  tests.  Mislevy  and  Stocking  (1987)  point  out  that  the  usefulness  of 
these  indices  appears  particularly  limited  in  situations  like  NAEP  where  examinees  have  been  administered 
relatively  short  tests.  A study  by  Stone,  Mislevy,  and  Mazzeo  (1994)  using  simulated  data  suggests  that  the 
correct  reference  chi-square  distributions  for  these  indices  have  considerably  fewer  degrees  of  freedom  than 
the  value  indicated  by  the  BILOG/PARSCALE  program  and  require  additional  adjustments  of  scale.  However, 
it  is  not  yet  clear  how  to  estimate  the  correct  number  of  degrees  of  freedom  and  necessary  scale  factor 
adjustment  factors.  Consequently,  pseudo  chi-square  goodness-of-fit  indices  are  used  only  as  rough  guides  in 
interpreting  the  severity  of  model  departures. 


Second,  as  discussed  in  Chapter  9,  it  is  almost  certainly  the  case  that,  for  most  items,  item-response 
models  hold  only  to  a certain  degree  of  approximation.  Given  the  large  samples  sizes  used  in  NAEP,  there  will 
be  sets  of  items  for  which  one  is  almost  certain  to  reject  the  hypothesis  that  the  model  fits  the  data  (since  the 
likelihood  of  rejecting  the  null  increases  with  sample  size)  even  though  departures  are  minimal  in  nature  or 
involve  kinds  of  misfit  unlikely  to  impact  on  important  model-based  inferences  about  student  achievement.  In 
practice,  it  is  always  wise  to  temper  statistical  decisions  with  judgments  about  the  severity  of  model  misfit  and 
the  potential  impact  of  such  misfit  on  final  results. 

In  making  decisions  about  excluding  items  from  the  final  scales,  a balance  was  sought  between  being 
too  stringent,  hence  deleting  too  many  items  and  possibly  damaging  the  content  representativeness  of  the  pool 
of  scaled  items,  and  too  lenient,  hence  including  items  with  model  fit  poor  enough  to  invalidate  the  types  of 
inferences  made  from  NAEP  results.  Items  that  clearly  did  not  fit  the  model  were  not  included;  however,  a 
certain  degree  of  misfit  was  tolerated  for  a number  of  items  included  in  the  final  scales. 

For  the  large  majority  of  the  items,  the  fit  of  the  model  was  extremely  good.  Figure  13-1  provides  a 
typical  example  of  what  the  plots  look  like  for  a dichotomously-scored  item  in  this  class  of  items.  The  plot  that 
is  shown  is  for  an  item  from  the  physical  science  scale.  In  the  plot,  the  y-axis  indicates  the  probability  of  a 
correct  response  and  the  x-axis  indicates  scale  score  level  (0).  The  crosses  show  estimates  of  the  conditional 
(on  0)  probability  of  a correct  response  that  do  not  assume  a logistic  form  (referred  to  subsequently  as 
nonlogistic-based  estimates).  The  sizes  of  the  crosses  are  proportional  to  the  estimated  density  of  the  theta 
distribution  at  the  indicated  value.  The  solid  curve  shows  the  estimated  theoretical  item  response  function.  The 
item  response  function  provides  estimates  of  the  conditional  probability  of  a correct  response  based  on  an 
assumed  logistic  form.  The  vertical  dashed  line  indicates  the  estimated  location  parameter  (b)  for  the  item  and 
the  horizontal  dashed  line  indicates  the  estimated  lower  asymptote  (c).  Also  shown  in  the  plot  are  the  actual 
values  of  the  item  parameter  estimates  (lower  right-hand  comer).  As  is  evident  from  the  plot,  the  ‘empirical’  or 
non-logistic-based  item  trace  is  in  extremely  close  agreement  with  the  model-based  item  response  function 
logistic  curve. 

Figure  13-2  provides  an  example  of  a plot  for  a 4-category  extended  constructed-response  item 
exhibiting  good  model  fit.  Like  the  plots  for  the  dichotomously-scored  multiple-choice  items,  this  plot  shows 
two  estimates  of  each  item  category  characteristic  curve,  one  set  that  does  not  assume  the  partial  credit  model 
(the  empirical  trace  shown  as  asterisks)  and  one  that  does  (the  theoretical  trace  shown  as  solid  curves).  The 
estimates  for  all  parameters  for  the  item  in  question  are  also  indicated  on  the  plot.  As  with  Figure  13-1,  the  two 
sets  of  estimates  agree  quite  well,  although  there  is  a slight  tendency  for  the  nonlogistic-based  estimates  for 
category  two  to  be  somewhat  higher  than  the  model-based  estimates  for  theta  values  less  than  1.  An  aspect  of 
Figure  13-2  worth  noting  is  the  large  proportion  of  examinees  that  responded  in  the  two  lowest  response 
categories  for  this  item^.  Such  results  were  typical  for  the  extended  constructed-response  items.  Substantial 
proportions  of  examinees  were  either  unable  or  unwilling  to  provide  even  minimally  adequate  answers  to  such 
items. 


As  discussed  above,  some  of  the  items  retained  for  the  final  scales  display  some  degree  of  model 
misfit.  Figures  13-3  (a  dichotomously-scored  multiple-choice  item)  and  13-4  (an  extended  constructed- 
response  item)  provide  typical  examples  of  such  items.  Note  that  in  Figure  13-4,  the  empirical  curve  lies  above 
the  theoretical  curve  in  the  lower  part  of  the  ability  scale  for  the  lowest  category,  but  below  the  theoretical 
curve  for  the  next  higher  category.  Combining  these  two  categories  would  have  improved  the  model  fit,  but  it 
was  judged  that  the  misfit  was  not  sufficiently  pronounced  in  this  case  to  warrant  such  collapsing.  In  general, 
good  agreement  between  empirical  and  theoretical  item  traces  were  found  for  the  regions  of  the  theta  scale  that 
includes  most  of  the  examinees.  Misfit  was  confined  to  conditional  probabilities  associated  with  theta  values  in 
the  tails  of  the  subject  ability  distributions. 


^ This  is  evidenced  by  the  relatively  large  size  of  the  asterisks  indicating  estimated  conditional  probabilities  for  these  two  categories. 
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Item  K049907  (grade  12,  earth  science  scale)  did  not  fit  in  a run  with  and  unconstrained  (to  normal) 
prior  and  with  all  the  adjustments  that  had  been  made  in  the  national  scaling  (Figure  13-5).  Categories  1 and  2 
were  combined  to  yield  a 0-1  dichotomous  item,  and  the  fit  improved  substantially  (Figure  13-6). 


Figure  13-1 

Plot*  Comparing  Empirical  and  Model-Based  Estimates  of  Item  Response  Functions 
for  Binary  Scored  (Multiple-Choice)  Items  Exhibiting  Good  Model  Fit 


*Asterisks  indicate  estimated  conditional  probabilities  obtained  without  assuming  a logistic  form;  the  solid 
curve  indicates  estimated  item  response  function  assuming  a logistic  form. 
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Figure  13-2 

Plot^  Cottipdring  Ettipiriccil  and  Model-Based  Estitnates  of  Item  Category  Characteristic  Curves 
for  a Polytomously  Scored  Item  Exhibiting  Good  Model  Fit 


^Asterisks  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the 
solid  curve  indicates  estimated  item  response  function  assuming  a model-based  form. 
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Figure  13-3 

Plot*  Comparing  Empirical  and  Model-Based  Estimates  of  Item  Response  Functions 
for  Binary-Scored  (Multiple-Choice)  Item  Exhibiting  Some  Model  Misfit 


*Asterisks  indicate  estimated  conditional  probabilities  obtained  without  assuming  a logistic  form;  the  solid 
curve  indicates  estimated  item  response  function  assuming  a logistic  form. 
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Figure  13-4 

Plot*  Comparing  Empirical  and  Model-Based  Estimates  of  Item  Category  Characteristic  Curves 
for  a Polytomously  Scored  Item  Exhibiting  Some  Model  Misfit 


^Asterisks  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the 
solid  curve  indicates  estimated  item  response  function  assuming  a model-based  form. 
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Figure  13-5 

Plot*  Comparing  Empirical  and  Model-Based  Estimates  of  Item  Category  Characteristic  Curves 
for  a Polytomously  Scored  Item  (K044 10 1)  Exhibiting  Poor  Model  Fit 


*Asterisks  indicate  estimated  conditional  probabilities  obtained  without  assuming  a model-based  form;  the 
solid  curve  indicates  estimated  item  response  function  assuming  a model-based  form. 
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Figure  13-6 

Plot  ^ Comparing  Empirical  and  Model-Based  Estimates  of  Item  Category  Characteristic  Curves 
for  Polytomously  Scored  Item  (K044I0I)  After  Collapsing  Categories  2 and  3 


^Asterisks  indicate  estimated  conditional  probabilities  obtained  without  assuming  a logistic  form;  the  solid 
curve  indicates  estimated  item  response  function  assuming  a logistic  form. 

Note:  When  the  number  of  alternatives  of  a constructed-response  item  equaled  zero,  the  item  was  scored  in 
only  two  categories. 


On  the  following  pages,  Table  13-16  lists  the  items  that  received  special  treatment  during  the  scaling 
process  Included  in  the  table  are  the  block  locations  and  item  numbers  for  the  items  that  were  combined  into 
cluster  items  as  well  as  for  those  that  were  excluded  from  the  final  scales.  At  grade  8,  all  items  received 
identical  special  treatment  in  the  development  of  the  1996  State  Assessment  scales.  No  other  items  in  either 
assessment  received  special  treatment.  The  IRT  parameters  for  the  items  included  in  the  science  assessment 

are  listed  in  Appendix  D. 
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Table  13-16 

Items  from  the  1996  Science  Assessment  Receiving  Special  Attention 


Grade 

NAEP 

ID 

Block/Item 

Number 

Field  of 
Science 

Disposition 

Reason 

4 

K031105 

S4-5 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,0,1 

Lack  of  Fit 

4 

K031107 

S4-7 

Physical 

science 

Collapsed  categories:  0, 1,2,3 
becomes  0,0, 1,2 

Lack  of  Fit 

4 

K031203 

S5-3 

Physical 

science 

Dropped 

Dependency 

4,8 

K031301 

S6-6 

Physical 

science 

Dropped 

Dependency 

4 

K034802 

S13-2 

Physical 

science 

Collapsed  categories  0,1,2 
become  0,0,1 

Lack  of  Fit 

4 

K031402 

S7-2 

Earth 

science 

Collapsed  categories  : 0,1,2 
become  0,0,1 

Lack  of  Fit 

4 

K031404 

S7-4 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

4 

K031407 

S7-7 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

4 

K034501 

S12-  10 

Earth 

science 

Collapsed  categories:  0, 1,2,3 
become  0, 1,2,2 

Zero 

Frequency 

4 

K040501 

S21  - 11 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,0,1 

Lack  of  Fit 

4 

K031001 

S3  - 1 

Life 

science 

Dropped  to  form  cluster  item 

Dependency 

4 

K031002 

S3 -2 

Life 

science 

Dropped  to  form  cluster  item 

Dependency 

4 

K031605 

S9-5 

Life 

science 

Dropped 

Lack  of  Fit 

4 

K031607 

S9-9 

Life 

science 

Collapsed  categories  0,1, 2,3 
become  0,1, 2,2 

Lack  of  Fit 

4 

K032501 

S10-9A 

Life 

science 

Dropped  to  form  cluster  item 

Dependency 

4 

K032502 

S10-9G 

Life 

science 

Dropped  to  form  cluster  item 

Dependency 

4 

K033501 

Sll  -9 

Life 

science 

Collapsed  categories  0,1,2 
become  0,0,1 

Lack  of  Fit 

(continued) 
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Table  13-16  (continued) 

Items  from  the  1996  Science  Assessment  Receiving  Special  Attention 


Grade 

NAEP 

ID 

Block/Item 

Number 

Field  of 
Science 

Disposition 

Reason 

4 

KZ34101 

S12-6 

Earth 

Science 

Dropped 

1 

4 

KZ34401 

S12-9 

Earth 

Science 

Dropped 

1 

4 

KZ34501 

S12-  10 

Earth 

Science 

Dropped 

1 

4 

KZ34502 

S12-  11 

Earth 

Science 

Dropped 

1 

8 

K040601 

S3  - 1 

Physical 

science 

Collapsed  categories  0,1,2 
become  0,0,1 

Lack  of  Fit 

8 

K040702 

S4-3 

Physical 

science 

Collapsed  categories;  0,1, 2,3 
become  0,1, 1,2 

Lack  of  Fit 

8 

K040705 

S4-4 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Zero 

Frequency 

8 

K031306 

S6-9 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

^ro 

Frequency 

8 

K043603 

Sll - 16 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Zero 

Frequency 

8 

K040711 

S4-  12 

Earth 

science 

Dropped  to  form  cluster  item 

Dependency 

8 

K040712 

S4- 13 

Earth 

science 

Dropped  to  form  cluster  item 

Dependency 

8 

K040803 

S5-2 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

8 

K040901 

S7-  1 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

8 

K040905 

S7-5 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,0,1 

Lack  of  Fit 

8,  12 

K049403 

S13-15 

Earth 

science 

Dropped  to  form  cluster  item 

Dependency 

8,  12 

K049404 

S13-  16 

Earth 

science 

Dropped  to  form  cluster  item 

Dependency 

' This  item  was  deleted  due  to  an  error  discovered  after  scaling  was  completed. 
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Table  13-16  (continued) 

Items  from  the  1996  Science  Assessment  Receiving  Special  Attention 


Grade 

NAEP 

ID 

Block/Item 

Number 

Field  of 
Science 

Disposition 

Reason 

8 

K044101 

. S20  - 5 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

8 

K044401 

S20-8 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

8 

K041306 

S8-6 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

8 

K031603 

S9-3 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,0,1 

Lack  of  Fit 

8. 

K031611 

S9-11 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Zero 

Frequency 

8 

K042602 

SIO-  15 

Life 

science 

Collapsed  categories:  0,1, 2,3 
become  0,1, 1,2 

Lack  of  Fit 

8 

K049301 

S13-  12 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Zero 

Frequency 

8 

K037001 

S15  - 1 

Life 

science 

Dropped 

Lack  of  Fit 

12 

K041306 

S8-6 

Life 

science 

Dropped 

Lack  of  Fit 

12 

K049502 

S3 -4 

Physical 

science 

Collapsed  categories:  0,1, 2,3,4 
become  0,0, 1,2,3 

Lack  of  Fit 

12 

K049503 

S3 -5 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K049602 

S4-2 

Physical 

science 

Collapsed  categories:  0,1, 2,3,4 
become  0,0, 1,2,2 

Lack  of  Fit 

12 

K049603 

S4-3 

Physical 

science 

Collapsed  categories:  0,1, 2,3,4 
becomie  0,1,2,3,3 

Lack  of  Fit 

12 

K049702 

S6-2 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K049703 

S6-4 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,0,1 

Lack  of  Fit 

12 

K049706 

S6-7 

Physical 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K058201 

S20-  1 

Physical 

science 

Dropped 

DIF 

(continued) 
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Table  13-16  (continued) 

Items  from  the  1996  Science  Assessment  Receiving  Special  Attention 


Grade 

NAEP 

ID 

Block/Item 

Number 

Field  of 
Science 

Disposition 

Reason 

12 

K040801 

S5-0 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,0, 1 

Lack  of  Fit 

12 

K049804 

S7-4 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K049810 

S7  - 10 

Earth 

science 

Collapsed  categories:  0, 1 ,2,3 
become  0,1, 2,2 

Lack  of  Fit 

12 

K089811 

S7- 11 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K049812 

S7- 12 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,0, 1 

Lack  of  Fit 

12 

K049903 

S9-3 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K049904 

S9-4 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12. 

K049907 

S9-7 

Earth 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K057201 

S20-2 

Earth 

science 

Dropped 

Lack  of  Fit 

12 

K049506 

S3 -6 

Life 

science 

Collapsed  categories:  0,1,2,3 
become  0, 1,2,2 

Zero 

Frequency 

12 

K041401 

S8-8 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,0, 1 

Lack  of  Fit 

12 

K041404 

S8- 11 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,1,1 

Lack  of  Fit 

12 

K053601 

S14- 13 

Life 

science 

Collapsed  categories:  0,1,2,3,4 
become  0,1, 2,3,3 

Lack  of  Fit 

12 

K054006 

S15-6 

Life 

science 

Collapsed  categories:  0,1,2 
become  0,0, 1 

Lack  of  Fit 
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13.6 


GENERATION  OF  PLAUSIBLE  VALUES 


13.6.1  Principal  Components  (NSWEEP  Program) 

Multivariate  plausible  values  were  generated  for  the  entire  age/grade  sample  using  the  multivariate 
conditioning  program  CGROUP  as  revised  by  Thomas  (1994).  This  procedure  employed  student  weights. 
Prior  to  1990,  selected  background  variables  were  used  for  conditioning.  However,  from  1990  to  the  present, 
principal  components  of  the  background  variables  have  been  used  as  conditioning  variables.  Almost  all  of  the 
background  variables  were  coded  as  0-1  contrasts,  so  no  standardization  took  place.  Principal  components  of 
these  contrasts  were  employed  to  remedy  problems  of  extreme  collinearity  among  some  of  the  original 
conditioning  variables.  The  principal  components  used  accounted  for  at  least  90  percent  of  the  variance  of  the 
original  conditioning  variables. 

Results  from  research  on  the  1990  Trial  State  Assessment  in  mathematics  suggests  that  using  a large 
subset  of  principal  components  will  yield  estimates  that  differ  only  slightly  from  those  obtained  using  the  full 
set  (Mazzeo,  Johnson,  Bowker,  & Fong,  1994).  Table  13-17  contains  a list  of  the  number  of  principal 
components  included  in  conditioning,  as  well  as  the  proportion  of  variance  accounted  for  by  the  conditioning 
model  for  each  grade. 


Table  13-17 

Proportion  of  Proficiency  Variance  Accounted  for  by  the  Conditioning  Model 
for  the  Science  Main  Assessment 


Grade 

Number  of 
Conditioning 
Contrasts* 

Number  of 
Principal 
Components' 

Proportion  of  Proficiency  Variance 
Physical  Eartb  Life 

Science  Science  Science 

4 

948 

317 

.64 

.57 

.59 

8 

1,041 

326 

.63 

.63 

.64 

12 

808 

290 

.71 

.71 

.70 

' Excluding  the  constant  term 


13.6.2  Conditioning  (CGROUP  Program) 

The  codings  of  the  original  science-specific  conditioning  variables,  before  principal  components  were 
calculated,  are  presented  in  Appendix  C.  NAEP-CGROUP  (described  in  Chapter  1 1)  creates  posterior 
distributions  of  proficiencies  by  combining  information  from  item  responses  of  individuals  and  information 
from  linear  regression  of  proficiency  on  conditioning  variables.  For  each  individual,  five  plausible  values  are 
randomly  drawn  from  their  posterior  proficiency  distribution. 

The  values  of  the  conditioning  effects  are  expressed  in  the  metrics  of  the  original  calibration  scale. 
Definitions  of  derived  conditioning  variables  are  given  in  Appendix  B. 


13.6.3  Analysis  of  Dimensionality 


As  mentioned  earlier,  the  main  assessment  is  multivariate  with  three  content  area  scales.  Tables  13-18 
and  13-19  give  conditional  and  marginal  correlations  for  the  three  scales  for  the  three  grades.  The  conditional 
correlations  can  be  thought  of  as  correlations  from  information  pooled  within  the  demographic  subgroups 
corresponding  to  grouping  variables  used  to  condition  the  data  with  CGROUP.  The  conditional  correlations 
correspond  to  the  error  correlations  of  a CGROUP  analysis.  The  conditional  correlations  are  high,  averaging 
.79  for  grade  4,  .90  for  grade  8,  and  .83  for  grade  12.  The  marginal  correlations  are  the  average  of  the  scale 
correlations  over  five  plausible  values  generated  by  CGROUP.  Since  these  correlations  are  not  pooled  within 
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background  groupings,  marginal  correlations  tend  to  be  larger  than  conditional  correlations,  averaging  .84  for 
grade  4,  .91  for  grade  8,  and  .90  for  grade  12.  Although  it  is  of  substantive  interest  to  analyze  the  scales 
separately,  the  correlations  indicate  that  they  are  highly  redundant  with  each  other. 


Table  13-18 

Conditional  Correlations  from  Conditioning  (CGROUP) 


Grade 

Field  of  Science  Scale 

Physical  Science 

Earth  Science 

Life  Science 

4 

Physical  Science 

1.00 

— 

— 

Earth  Science 

0.79 

1.00 

— 

Life  Science 

0.79 

0.78 

1.00 

8 

Physical  Science 

1.00 

— ■ 

— 

Earth  Science 

0.92 

1.00 

— 

Life  Science 

0.89 

0.88 

1.00 

12 

Physical  Science 

1.00 

— 

— 

Earth  Science 

0.87 

1.00 

— 

Life  Science 

0.82 

0.80 

1.00 

Table  13-19 

Marginal  Correlations  of  Science  Scales^ 

Grade 

Field  of  Science  Scale 

Physical  Science  Earth  Science 

Life  Science 

4 

Physical  Science 

1.00 

— 

— 

Earth  Science 

0.84 

1.00 

— 

Life  Science 

0.84 

0.84 

1.00 

8 

Physical  Science 

1.00 

— 

— 

Earth  Science 

0.93 

1.00 

— 

Life  Science 

0.91 

0.90 

1.00 

12 

Physical  Science 

1.00 

— 

— 

Earth  Science 

0.92 

1.00 

— 

Life  Science 

0.90 

0.89 

1.00 

‘ Tabled  values  were  obtained  by  computing  a separate  Pearson  correlation  coefficient  for  each  plausible  value,  computing  Fisher’s 
z-transformation  for  each  value,  computing  the  average  of  the  transformed  values,  and  computing  the  inverse  transformation  of  the 
average. 


13.7  THE  FINAL  PROFICIENCY  SCALES 
13.7.1  Field  of  Science  Scales 

Like  all  IRT  scales,  the  field  of  science  scales  have  a linear  indeterminacy  that  may  be  resolved  by  an 
arbitrary  choice  of  origin  and  unit-size  for  each  scale.  The  1996  science  assessment  was  developed  using  a 
new  framework.  Because  it  was  not  appropriate  to  compare  results  from  the  1996  assessment  with  those  of 
previous  NAEP  science  assessments,  no  attempt  was  made  to  link  or  align  scores  on  the  new  assessment  to 
those  of  previous  assessments.  Therefore,  it  was  necessary  to  establish  a new  scale  for  reporting.  NAEP 
assessments  developed  earlier  (such  as  the  1992  reading  assessment)  were  developed  with  a cross-grade 
framework,  in  which  the  trait  being  measured  is  conceptualized  as  cumulative  across  the  grades  of  the 
assessment.  Accordingly,  a single  O-to-500  scale  was  established  for  all  three  grades  in  each  of  these 
assessments. 


O 
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In  1993,  the  National  Assessment  Governing  Board  (NAGB)  determined  that  future  NAEP 
assessments  should  be  developed  using  within-grade  frameworks.  This  removes, the  constraint  that  the  trait 
being  measured  is  cumulative.  It  also  means  that  there  is  no  need  for  overlap  items  across  grades.  Consistent 
with  this  view,  NAGB  also  declared  that  scaling  be  performed  within-grade.  Any  items  that  happened  to  be  the 
same  across  grades  in  the  assessment  were  scaled  separately  for  each  grade,  thus  making  it  possible  for 
common  items  to  function  differently  in  the  separate  grades.  The  NAEP  1994  U.S.  history  and  geography 
assessments  were  developed  and  scaled  within-grade.  After  scaling,  the  scales  were  aligned  so  that  grade  8 had 
a higher  mean  than  grade  4,  and  grade  12  had  a higher  mean  than  grade  8.  The  results  were  reported  on  a final 
O-to-500  scale  that  looked  similar  to  those  used  in  reading,  in  spite  of  the  differences  in  development  and 
scaling.  This  choice  of  the  reporting  scale  was  the  source  of  potential  confusion  and  misinterpretation. 

Therefore,  for  the  NAEP  1996  science  assessment — which  was  also  developed  and  scaled  using 
within-grade  procedures — a new  reporting  metric  was  adopted.  The  results  are  reported  on  O-to-300  scales  and 
the  means  for  each  of  the  grades  are  identical.  For  each  grade,  the  mean  for  each  field  of  science  was  set  at  150 
and  the  standard  deviation  was  35.  Constraining  the  mean  and  standard  deviation  of  the  scales  to  150  and  35 
also  constrained,  to  some  degree,  the  percentiles  for  the  total  group  of  students  at  each  grade.  However, 
within-grade  comparisons  of  percentiles  across  subgroups  can  still  provide  valuable  comparative  information. 
The  reporting  metric  was  developed  using  data  from  the  national  assessment  program,  and  the  results  for  the 
state  assessment  program  were  linked  to  these  scales. 


For  each  field  of  science,  the  scale  mean  and  standard  deviation  were  set  to  150.0  and  35.0  using  the 
transformation: 

^target  = A • Ocalibrated  B. 

where  0u,rget  denotes  values  on  the  final  transformed  scale  and  Gcaiibrated  denotes  values  on  the  original 
calibration  scale  from  BILOG/PARSCALE.  The  calculation  of  the  value  of  “A”  and  “B”  is  described 
in  Chapter  9,  Section  9.3.5.  The  constants  for  the  linear  transformation  for  each  scale  are  given  in 
Table  13-20. 


Table  13-20 

Coefficients  of  Linear  Transformations  of  the  Fields  of  Science  Scales 
from  the  Calibrating  Scale  Units  to  the  Units  of  Reporting  Proficiency 


Grade 

Field  of  Science  Scale 

A 

B 

4 

Physical  Science 

34.91 

151.17 

Earth  Science 

34.09 

150.67 

Life  Science 

35.09 

150.51 

8 

Physical  Science 

35.85 

150.23 

Earth  Science 

34.56 

150.58 

Life  Science 

35.64 

150.25 

12 

Physical  Science 

37.76 

149.65 

Earth  Science 

36.59 

149.77 

Life  Science 

35.91 

150.19 

13.7.2  The  Composite  Science  Proficiency  Scale 

In  addition  to  the  plausible  values  for  each  scale,  a composite  of  the  three  fields  of  science  scales  was 
created  as  a measure  of  overall  science  performance.  The  composite  was  a weighted  average  of  plausible 
values  on  the  fields  of  science  scales  (earth,  physical,  and  life).  The  weights  for  the  scales  were  proportional  to 
the  importance  assigned  to  each  field  of  science  contained  in  the  assessment  specifications  given  in  the  Science 
Framework.  The  weights  are  given  in  Table  13-21.  As  indicated  in  Table  2-4  of  Chapter  2,  the  weights  for 
each  of  the  fields  of  science  are  similar  to  the  actual  proportion  of  assessment  time  devoted  to  that  field.  In 
O 
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developing  the  composite  scale,  the  weights  were  applied  to  the  plausible  values  for  each  fields  of  science  as 
expressed  in  terms  of  the  final  scale  (i.e.,  after  transformation  from  the  provisional  BILOG/P ARSCALE 
scales).-  . , 


Table  13-21 

Weights  Used  for  Each  Field  of  Science  Scale  to  Form  the  Science  Composite 


Field  of  Science  Scale 

Grade  4 

Grade  8 

Grade  12 

Physical  science 

■ .33 

.30 

.33 

Earth  science 

.33 

.30 

.33 

Life  science 

.33 

.40 

.33 

Finally,  it  is  necessary  to  caution  that,  although  the  science  composite  is  expressed  in  units  that  seem 
somewhat  similar  to  the  long-term  trend  science  scale,  it  is  not  appropriate  to  compare  scores.  The 
transformation  chosen  to  resolve  the  linear  indeterminacy  in  the  science  composite  is  a convenient 
transformation,  but  it  is  only  one  of  a conceptually  infinite  number  of  such  transformations  that  could  have 
been  chosen.  Any  one  of  these  transformations  would  have  provided  equivalent  information  about  the  relative 
standings  of  subgroups  in  the  population.  Because  there  is  no  link,  real  or  implied,  in  the  construction  of  the 
science  composite  and  the  field  of  science  scales  to  either  the  mathematics  assessment  or  to  the  previous 
science  assessments,  the  comparison  of  students’  science  proficiencies  to  students’  proficiencies  in  other 
subject  areas  is  devoid  of  meaning. 


13.8  PARTITIONING  OF  THE  ESTIMATION  ERROR  VARIANCE 

For  each  grade,  the  error  variance  of  the  final,  transformed  scale  mean  was  partitioned  as  described  in 
Chapter  11.  This  analysis  yielded  estimates  of  the  proportion  of  error  variance  due  to  sampling  students  ^d 
the  proportion  due  to  the  latent  nature  of  0.  These  estimates  are  given  in  Table  13-22  for  each  field  of  science 
scale  and  the  composite  scale  (for  stability,  the  estimates  of  the  between-imputation  variance,  B,  in  Equation 
1 1.9).  Additional  results,  including  those  by  gender  and  race/ethnicity,  are  presented  in  Appendix  E. 


Table  13-22 

Estimation  Error  Variance  and  Related  Coefficients  for  the  Science  Main  Assessment 


Grade 

Field  of  Science  Scale 

Total 

Estimation 

Error 

Variance 

Proportion  of  Variance 
Due  to  . . . 

Student 

Sampling  Latency  of  6 

4 

Physical  Science 

1.16 

0.82 

0.18 

Earth  Science 

0.72 

0.85 

0.15 

Life  Science 

0.86 

0.78 

0.22 

Composite 

0.64 

0.89 

0.11 

8 

Physical  Science 

0.91 

0.92 

0.08 

Earth  Science 

0.89 

0.91 

0.09 

Life  Science 

, 1.07 

0.86 

0.14 

Composite 

0.78 

0.94 

0.06 

12 

Physical  Science 

1.03 

0.92 

0.08 

Earth  Science 

0.91 

0.94 

0.06 

^ Life  Science 

0.80 

0.89 

0.11 

Composite 

0.76 

0.96 

0.04 
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13.9  SCIENCE  TEACHER  QUESTIONNAIRES 


Teachers  of  fourth-  and  eighth-grade  students  were  surveyed  about  their  educational  background  and 
teaching  practices.  The  students  in  a particular  classroom  had  their  records  matched  with  their  teacher’s  survey 
information.  Variables  derived  from  the  questionnaire  were  used  in  the  conditioning  models,  along  with  a 
variable  that  indicated  whether  a student  record  had  been  matched  with  a teacher  record,  which  controls 
estimates  of  subgroup  means  for  differences  that  exist  between  the  matching  and  non-matching  students.  Of 
the  7,305  fourth-grade  students  in  the  sample,  89.0  percent  were  matched  with  both  parts  of  the  teacher 
questionnaire  and  2.2  percent  were  matched  with  only  the  first,  teacher  background,  part  of  the  questionnaire. 
For  the  eighth-grade  students  sample,  82.4  percent  were  matched  with  both  parts  of  the  teacher  questionnaire 
and  1.4  percent  were  matched  with  only  the  first  part  of  the  questionnaire.  The  lower  match  rate  for  both  parts 
of  the  questionnaire  for  eighth-grade  students  was  due  in  part  to  the  fact  that  in  grade  8 students  were  matched 
to  the  particular  class  that  the  teacher  taught.  Class  membership  information  was  often  missing  or  ambiguous. 
For  grade  4,  students  only  had  to  be  matched  to  the  teacher,  resulting  in  higher  match  rates.  Thus,  91,4  percent 
of  the  fourth  graders  and  83.8  percent  of  the  eighth  graders  were  matched  with  at  least  the  background 
information  about  their  science  teachers. 


13.10  THE  WEIGHT  FILES 

Westat  produced  the  final  student  and  school  weights  and  the  corresponding  replicate  weights  for  the 
1996  science  assessment.  Information  for  the  creation  of  the  weight  files  was  supplied  by  NCS  under  the 
direction  of  ETS. 

As  was  described  in  the  Technical  Report  of  the  1996  State  Assessment  Program  in  Science  (Allen, 
Swinton,  Isham,  & Zelenak,  1998),  the  State  Assessment  sample  was  split  into  two  subsamples,  one  using  the 
1992  inclusion  rules  (SI)  and  one  using  the  1996  inclusion  rules  (S2)  the  weighting  process  was  more  complex 
than  in  previous  assessments  (see  Allen,  Swinton,  Isham,  & Zelenak,  1998  for  more  details). 

In  the  national  science  assessment,  only  the  1996  inclusion  rules  (S2)  were  used.  Also,  there  were  no 
accommodations  offered  for  students  with  disabilities  or  for  students  with  limited  English  proficiency  in  the 
national  science  assessment.  Thus,  a single  sample  was  used  for  both  analysis  and  reporting,  and  only  a single 
set  of  student  sampling  weights.  The  student  sampling  weights  have  replicate  weights  associated  with  them. 
Replicate  weights  are  used  to  estimate  jackknife  standard  errors  for  each  statistic  estimated  for  the  national 
science  assessment. 


Chapter  14 


DATA  ANALYSIS  FOR  THE  LONG-TERM  TREND 
READING  ASSESSMENT^ 

Jo-Lin  Liang  and  Lois  H.  Worthington 
Educational  Testing  Service 


14.1  INTRODUCTION 

This  chapter  describes  the  analyses  performed  on  the  responses  to  the  cognitive  and  background 
items  in  the  1996  long-term  trend  reading  assessment.  These  analyses  led  to  the  results  presented  in  the 
NAEP  1996  Trends  in  Academic  Progress:  Achievement  of  U.S.  Students  in  Science,  1969  to  1996; 
Mathematics,  1973  to  1996;  Reading,  1971  to  1996;  and  Writing,  1984  to  1996  (Campbell,  Voelkl,  & 
Donahue,  1997).  The  emphasis  of  this  chapter  is  on  the  methods  and  results  of  procedures  used  to 
develop  the  IRT-based  scale  scores  that  formed  the  basis  of  this  report.  The  theoretical  underpinnings  of 
the  IRT  and  plausible  values  methodology  described  in  this  chapter  are  given  in  Chapter  11,  and  several 
of  the  statistics  are  described  in  Chapter  9. 

The  objectives  of  the  reading  long-term  trend  analysis  were  to  prepare  scale  values  and  perform 
all  analyses  necessary  to  produce  a long-term  trend  report  in  reading.  The  reading  long-term  trend  results 
include  the  years  1971,  1975,  1980,  1984,  1988,  1990,  1992,  1994,  and  1996.  The  major  analysis 
components  are  discussed  in  turn.  Some  aspects  of  the  analysis,  such  as  procedures  for  item  analysis, 
scoring  of  constructed-response  items,  and  methods  of  scaling,  are  described  in  previous  chapters  and  are 
therefore  not  detailed  here. 

The  student  samples  that  were  administered  reading  items  in  the  1996  long-term  trend  reading 
assessment  are  shown  in  Table  14-1.  See  Chapters  1 and  3 for  descriptions  of  the  target  populations  and 
the  sample  design  used  for  the  assessment. 

The  long-term  trend  results  reported  in  Trends  in  Academic  Progress  are  based  on  print 
administrations  and  occur  at  all  of  the  age  levels.  The  samples  involved  in  the  analysis  are  shown  in 
Table  14-1  as  9[RW-LTTrend],  13[RW-LTTrend],  and  17[RW-LTTrend].  The  long-term  trend  booklets 
for  these  samples  contained  blocks  of  reading  and  writing  items  administered  in  print  form.  All  students 
received  a block  of  common  background  questions,  distinct  for  each  age,  and  subject-area  background 
questions  that  were  presented  in  the  cognitive  blocks.  The  booklets  are  identical  to  those  used  for  long- 
term trend  assessments  in  1984,  1988,  1990,  1992,  and  1994.  The  booklets  and  the  blocks  within  those 
booklets  are  listed  in  Chapter  4.  Additional  information  about  all  of  the  items  in  these  blocks  also  appears 
in  that  chapter.  This  chapter  includes  specific  information  about  the  long-term  trend  items  that  were 
scaled.  Both  age-  and  grade-selected  students  contributed  to  the  long-term  trend  scaling.  However,  only 
students  in  the  “age-only”  portion  of  the  reading  long-term  trend  samples  contributed  to  the  results 
presented  in  Trends  in  Academic  Progress. 


' Jo-Lin  Liang  was  the  primary  person  responsible  for  the  planning,  specification,  and  coordination  of  the  reading  long-term 
trend  analyses,  advised  by  Eiji  Muraki,  and  Nancy  L.  Allen.  Data  analyses  and  scaling  were  performed  by  Lois  H.  Worthington, 
advised  by  David  S.  Freund.  Others  contributing  to  the  analysis  of  data  were  Bruce  A.  Kaplan,  and  Norma  A.  Norris.  John  R. 
Donoghue  provided  consultation. 
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Table  14-1 

NAEP  1996  Long-Term  trend  Reading  Student  Samples 


Sample 

Booklet 

IDs 

Mode 

Cohort 

Assessed 

Time  of 
Testing 

Age 

Definition 

Modal 

Grade 

Number 

Assessed 

9 [RW-LTTrend] 

51-56 

Print 

Age  9/Grade  4 

1/3/96  - 3/8/96  (Winter) 

CY 

4 

5,019 

13  [RW-LTTrend] 

51-56 

Print 

Age  13/Grade  8 

10/9/95  - 12/22/95  (Fall) 

CY 

8 

5,493 

17  [RW-LTTrend] 

51-56 

Print 

Age  17/Grade  11 

3/11/96-5/10/96  (Spring) 

Not  CY 

11 

4,669 

LEGEND 

RW  Reading  and  writing 

LTTrend  Long-term  trend  assessment 
Print  Print  administration 

CY  Calendar  year:  birthdates  in  1986,  and  1982  for  ages  9,  and  1 3 

Not  CY  Age  17  only:  birthdates  between  October  1,  1978,  and  September  30,  1979 

Table  14-2  clarifies  the  relationships  between  the  1996  long-term  trend  samples  and  samples 
from  previous  years.  For  ages  9,  13,  and  17,  the  [RW-LTTrend]  samples  allow  direct  comparisons  with 
1994,  1992,  1990,  1988,  and  1984  samples.  The  long-term  trend  scale,  established  in  1984,  was  linked  to 
the  1971,  1975,  and  1980  assessments  using  a complex  equating  strategy  described  in  Implementing  the 
New  Design:  The  NAEP  1983-84  Technical  Report  (Beaton,  1987).  At  each  age,  several  intact  booklets 
were  retained  from  the  1984  assessment,  forming  the  basis  of  the  reading  long-term  trend  assessment  in 
1988,  1990,  1992,  1994,  and  1996.  Information  about  the  1988  assessment  is  available  in  Focusing  the 
New  Design:  The  NAEP  1988  Technical  Report  (Johnson  & Zwick,  1990);  information  about  the  1990 
assessment  is  given  in  The  NAEP  1990  Technical  Report  (Johnson  & Allen,  1992);  information  about  the 
1992  assessment  is  given  in  The  NAEP  1992  Technical  Report  (Johnson  & Carlson,  1994);  and 
information  about  the  1994  assessment  is  given  in  The  NAEP  1994  Technical  Report  (Allen,  Kline,  & 
Zelenak,  1996). 

The  1996  long-term  trend  included,  at  each  age  level,  six  of  the  assessment  booklets 
administered  in  1984.  These  booklets  (51-56)  contained  both  reading  and  writing  blocks,  as  well  as 
background  items.  Although  these  long-term  trend  booklets  represented  only  about  one-tenth  of  the 
reading  booklets  administered  in  the  complex  1984  BIB  design,^  they  contained  10  of  the  12  reading 
blocks  that  were  scaled  at  each  age/grade  level  in  1984.  The  samples  of  students  who  received  these 
long-term  trend  booklets  are  described  in  Table  14-1  and  in  Chapter  3.  The  purpose  of  the  reading  long- 
term trend  analysis  was  to  add  to  the  reading  trend  results  that  extended  from  1971  to  1994  for  ages  9, 

13,  and  17.  The  numbers  of  scaled  items  for  each  age  are  presented  in  Table  14-3.  Each  age  was  scaled 
separately.  The  numbers  of  items  scaled  in  1996  that  were  common  across  assessment  years  are  given  in 
Table  14-4.  As  was  the  case  for  previous  long-term  trend  analyses,  the  long-term  trend  scale  is  univariate. 
Dimensionality  analyses  conducted  following  the  1984  assessment  showed  that  the  reading  items  were 
well  summarized  by  a unidimensional  scale  (Zwick,  1987). 


^ The  long-term  trend  assessment  included  1984  Booklets  16,  17,  27,  34,  55,  and  60  at  age  9 and  Booklets  13,  16,  17,  21,  34,  and 
57  at  ages  13  and  17  (see  J.  R.  Johnson,  1987,  pp.  120-121).  The  1984  main  assessment  focused-BIB  design  included  57 
booklets  that  contained  at  least  one  scaled  reading  block  at  age  9 and  56  such  booklets  at  ages  13  and  17. 
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Table  14-2 

NAEP  Reading  Samples  Contributing  to  1996  Long-Term  Trend  Results,  1971-1996 


Time  of 

Mode  of 

Age 

Modal 

Cohort 

Year 

Sample 

Subjects 

Testing 

Administration 

Definition 

Grade 

Age  9 

1971 

Main 

RL 

Winter 

Tape 

CY 

4 

1975 

Main 

RA 

Winter 

Tape 

CY 

4 

1980 

Main 

RA 

Winter 

Tape 

CY 

4 

1984 

Main 

RW 

Winter,  Spring 

Print 

CY 

4 

1984 

T-84 

RW 

Winter 

Tape 

CY 

4 

1988 

LTTrend' 

RW 

Winter 

Print 

CY 

4 

1990 

LTTrend' 

RW 

. Winter 

Print 

CY 

4 

1992 

LTTrend' 

RW 

Winter 

Print 

CY 

4 

1994 

LTTrend' 

RW 

Winter 

Print 

CY 

4 

1996 

LTTrend' 

RW 

Winter 

Print 

CY 

4 

Age  13 

1971 

Main 

RL 

Fall 

Tape 

CY 

8 

1975 

Main 

RA 

Fall 

Tape 

CY 

8 

1980 

Main 

RA 

Fall 

Tape 

CY 

8 

1984 

Main 

RW 

Winter,  Spring 

Print 

CY 

8 

1984 

T-84 

RW 

Fall 

Tape 

CY 

8 

1988 

LTTrend' 

RW 

Fall 

Print 

CY 

8 

1990 

LTTrend' 

RW 

Fall 

Print 

CY 

8 

1992 

LTTrend' 

RW 

Fall 

Print 

CY 

8 

1994 

LTTrend' 

RW 

Fall 

Print 

CY 

8 

1996 

LTTrend' 

RW 

Fall 

Print 

CY 

8 

Age  17 

1971 

Main 

RL 

Spring 

Tape 

NotCY 

11 

1975 

Main 

RABS 

Spring 

Tape 

NotCY 

11 

1980 

Main 

RA 

Spring 

Tape 

NotCY 

11 

1984 

Main 

RW 

Winter,  Spring 

Print 

NotCY 

11 

1984 

T-84 

RW 

Spring 

Tape 

Not  CY 

11 

1988 

LTTrend' 

RW 

Spring 

Print 

Not  CY 

11 

1990 

LTTrend' 

RW 

Spring 

Print 

NotCY 

11 

1992 

LTTrend' 

RW 

Spring 

Print 

NotCY 

11 

1994 

LTTrend' 

RW 

Spring 

Print 

NotCY 

11 

1996 

LTTrend' 

RW 

Spring 

Print 

NotCY 

11 

*Note:  Within  a cohort,  these  samples  received  common  booklets. 


LEGEND 


RL 

Reading  and  literature 

RA 

Reading  and  art 

RABS 

Reading,  art,  index  of  basic  skills 

RW 

Reading  and  writing 

Main 

Main  assessment 

T-84 

Special  sample  in  the  1984  assessment  that  was 
used  to  establish  links  to  previous  assessments 
(1971-1980)  for  the  purposes  of  long-term  trend 

LTTrend 

Long-term  trend  (these  samples  received 
common  booklets  within  an  age  group) 

Print 

Print  administration 

Tape 

Audiotape  administration 

CY 

Calendar  year:  birthdates  (1996  sample)  in  1986 
and  1 982  for  ages  9 and  1 3 

Not  CY 

Age  17  only  (1996  sample):  birthdates  between 
October  1 and  September  30  of  the  appropriate 

years 

i. 
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Table  14-3 

Numbers  of  Scaled  Reading  Long-Term  Trend 
Items  Common  Across  Ages 


Age 

Number  of  Items 

9 only 

61' 

13  only 

22' 

17  only 

23 

9 and  13  only 

13 

9 and  17  only 

2 

13  and  17  only 

42 

9,  13,  and  17 

26' 

Total 

189' 

' These  figures  have  been  updated  since  their 
publication  in  the  1992  and  1994  NAEP  technical 
reports  (Table  12-3  and  Table  15-3,  respectively). 


Table  14-4 

Numbers  of  Scaled  Reading  Long-Term  Trend  Items  Common  Across  Assessments 


Number  of  Items 


Assessment  Year 

Age  9 

Age  13 

Age  17 

1984, 

1992, 1994,  1996 

102 

103 

93' 

1984, 

1990,  1992,  1994,  1996 

101 

101 

92' 

1984, 

1988,  1990,  1992,  1994,  1996 

98 

98 

87 

1980, 

1984,  1988,  1990,  1992,  1994,  1996 

67 

71 

52 

1971, 

1975,  1980,  1984,  1988,  1990,  1992,  1994,  1996 

36 

45 

37 

' These  figures  have  been  updated  since  their  publication  in  the  NAEP  1992,  and  1994  Technical  Reports 
(Table  12-4,  and  Table  15-4,  respectively). 


The  steps  in  the  reading  long-term  trend  analysis  are  documented  in  the  following  sections.  As  is 
usual  in  NAEP  analyses,  the  first  step  was  to  gather  item  and  block  information.  The  trend  items  were 
then  calibrated  according  to  the  IRT  model.  Plausible  values  were  generated  after  conditioning  on 
available  background  variables.  Finally,  the  scale  values  were  placed  on  the  final  reading  long-term  trend 
scale  used  in  previous  trend  assessments. 


14.2  ITEM  ANALYSIS  FOR  THE  READING  LONG-TERM  TREND  ASSESSMENT 

Conventional  item  analyses  did  not  identify  any  difficulties  with  the  long-term  trend  data.  The 
results  displayed  in  Table  14-5  contain  the  number  of  items,  size  of  the  unweighted  sample  administered 
the  block,  average  weighted  proportion  correct,  average  weighted  r-biserial,  and  average  weighted  alpha 
as  a measure  of  reliability  for  each  block.  Because  the  blocks  were  presented  in  self-paced,  print- 
administered  form,  the  weighted  proportion  of  students  attempting  the  last  item  is  included  in  the  table  to 
give  an  indication  of  the  speededness  of  each  block.  Common  labeling  of  these  blocks  across  ages  does 
not  denote  common  items.  Booklet  information  is  detailed  in  Chapter  4.  Student  weights  were  used  for 
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all  statistics  except  for  the  sample  sizes.  The  average  values  reflect  only  the  items  in  the  block  that  were 
scaled.  Overall,  the  1996  item-level  statistics  were  not  very  different  from  those  for  the  1994  assessment. 


Table  14-5 

Descriptive  Statistics  for  Item  Blocks  in  the  Reading  Long-Term  Trend  Samples 


Statistics 

B8 

BIO 

Bll 

Bll 

B13 

Blocks 

B14 

B15 

B16 

B17 

B18 

Bll 

Age  9 

Number  of  scaled  items 

10 

8 

11 

7 

11 

12 

11 

1 

11 

12 

9 

Number  of  scaled  , 

constructed-response  items 

1 

0 

0 

1 

1 

1 

0 

1 

0 

0 

3 

Unweighted  sample  size 

630 

610 

610 

603 

624 

624 

595 

1 

603 

1222 

592 

Average  weighted  proportion 
correct 

.62 

.43 

.43 

.50 

.41 

.60 

.51 

1 

.56 

.47 

.60 

Average  weighted  r-biserial 

.75 

.62 

.62 

.82 

.61 

.75 

.60 

1 

.74 

.64 

.73 

Weighted  alpha  reliability 

.75 

.67 

.71 

.76 

.71 

.84 

.64 

1 

.81 

.75 

.74 

Weighted  proportion  of  students 
attempting  last  item 

.92 

.82 

.77 

.72 

.66 

.66 

.86 

1 

.86 

.82 

.97 

Age  13 

Number  of  scaled  items 
Number  of  scaled 

12 

9 

8 

5 

11 

12 

10 

9 

16 

11 

constructed-response  items 

1 

0 

0 

0 

1 

1 

1 

1 

0 

0 

Unweighted  sample  size 
Average  weighted  proportion 

629 

642 

630 

673 

630 

615 

658 

642 

673 

629 

correct 

.64 

.59 

.63 

.64 

.58 

.66 

.66 

.73 

.59 

.67 

Average  weighted  r-biserial 

.71 

.59 

.73 

.80 

.65 

.68 

.64 

.77 

.56 

.74 

Weighted  alpha  reliability 
Weighted  proportion  of  students 

.71 

.60 

.68 

.57 

.66 

.78 

.55 

.67 

.70 

.77 

attempting  last  item 

.94 

.83 

.98 

.95 

.91 

.81  , 

.84 

.92 

.78 

.97 

Age  17 

Number  of  scaled  items 
Number  of  scaled 

12 

4 

8 

6 

11 

12 

13 

10 

10 

7 

constructed-response  items 

1 

1 

0 

1 

1 

1 

1 

1 

0 

0 

Unweighted  sample  size 
Average  weighted  proportion 

605 

638 

585 

643 

585 

588 

622 

638 

643 

605 

correct 

.72 

.72 

.79 

.72 

.68 

.84 

.65 

.74 

.53 

.68 

Average  weighted  r-biserial 

.75 

.78 

.83 

.81 

.76 

.78 

.62 

.71 

.63 

.79 

Weighted  alpha  reliability 
Weighted  proportion  of  students 

.72 

.43 

.67 

.41 

.68 

.80 

.73 

.74 

.70 

.70 

attempting  last  item 

.95 

.97 

1.00 

.96 

.96 

.89 

.69 

.82 

.72 

.98 

^ Block  B 1 6 was  not  administered  at  age  class  9. 

2 

Block  B22  was  not  administered  at  age  class  13  or  17. 


14.3  TREATMENT  OF  CONSTRUCTED-RESPONSE  ITEMS 


Data  for  constructed-response  items  in  the  long-term  trend  analysis  were  used  for  the  1984,  1990, 
1992,  1994,  and  1996  assessments  only.  Constructed-response  items  were  not  included  in  the  original 
scoring  of  the  1988  reading  assessment  because  a previous  study  (Zwick,  1988)  had  shown  that  scoring 
inconsistencies  (drops  in  interrater  reliability  and/or  scorer  drift — that  is,  scorers  showed  evidence  of 
rating  items  more  strictly  or  more  leniently  than  did  the  original  1984  scorers)  had  affected  these  items. 

Rater  reliability  within  year  was  computed  for  the  1996  constructed-response  items  at  each  age. 
Between  year  reliability  was  also  studied  with  the  1994,  and  the  1984  responses.  In  general,  the  1996 
scoring  did  not  show  irregularities.  All  of  the  1996  constructed-response  items  were  used  in  the  trend 
analysis  except  the  items  that  were  excluded  from  calibration  in  the  previous  assessment.  The  deleted 
items  are  listed  in  Table  14-6.  The  remaining  constructed-response  items  were  dichotomized  according  to 
criteria  developed  by  subject-area  experts.  The  dichotomized  versions  of  the  constructed-response  items 
were  included  in  the  calibration. 


Table  14-6 

Items  Deleted  from  the  Reading  Long-Term  Trend  Analysis 


Age 

Block 

Item 

Reason  for  Exclusion 

9 

BIO 

N001801 

Excluded  in  previous  assessment 

B13 

N003003 

Excluded  in  previous  assessment 

BIO 

N008905 

Excluded  in  previous  assessment  (constructed-response  item) 

13 

BIO 

N001801 

Excluded  in  previous  assessment 

BIO 

N001904 

Excluded  in  previous  assessment  (constructed-response  item) 

Bll 

N002302 

Excluded  in  previous  assessment 

B12 

N002804 

Excluded  in  previous  assessment  (constructed-response  item) 

B17 

N005001 

Excluded  in  previous  assessrnent 

17 

BIO 

N001702 

Excluded  in  previous  assessment 

Bll 

N002302 

Excluded  in  previous  assessment 

B17 

NO 15905 

Excluded  in  previous  assessment  (constructed-response  item) 

14.4  IRT  SCALING  FOR  THE  READING  LONG-TERM  TREND  ASSESSMENT 
14.4.1  Item  Parameter  Estimation 

The  first  step  in  the  scaling  process  was  the  estimation  of  item  parameters  for  the  long-term  trend 
items.  This  item  calibration  was  performed  using  the  BILOG/PARSCALE  program  described  in 
Chapter  11.  Items  were  calibrated  separately  for  each  of  the  three  age/grade  groups.  Item  parameters 
were  estimated  using  combined  data  from  the  assessment  years  1994  and  1996,  treating  each  assessment 
as  a sample  from  a separate  subpopulation.  Student  weights  were  used  for  the  calibration.  To  ensure  that 
each  assessment  year  had  a similar  influence  on  the  calibration,  student  weights  for  1994  examinees  were 
multiplied  by  a constant,  to  adjust  them  to  have  the  same  sum  as  the  sum  of  the  weights  for  the  1996 
examinees.  Approximately  600-700  examinee  responses  for  each  item  were  present  in  each  assessment 
year. 
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Starting  values  for  item,  parameters  were  based  on  the  final  item  parameter  values  from  the 
analysis  of  the  1994  long-term  trend  assessment.  As  described  in  Chapter  9,  BILOG/PARSCALE 
calibrations  were  completed  in  two  stages.  At  stage  one,  the  proficiency  distribution  of  each  assessment 
year  was  constrained  to  be  normal,  although  the  means  and  variances  differed  across  assessment  years. 
The  values  of  the  item  parameters  from  this  normal  solution  were  then  used  as  starting  values  for  a 
second-stage  estimation  run  in  which  the  proficiency  distribution  (modeled  as  a separate  multinomial 
distribution  for  each  assessment  year)  was  estimated  concurrently  with  item  parameters.  Calibration  was 
concluded  when  changes  in  item  parameters  became  negligibly  small  (i.e.,  less  than  .005). 


14.4.2  Evaluation  of  Model  Fit 

During  and  subsequent  to  item  parameter  estimation,  evaluations  of  the  fit  of  the  IRT  models 
were  carried  out  for  each  of  the  items.  These  evaluations  were  based  primarily  on  graphical  analysis. 
First,  model  fit  was  evaluated  by  examining  plots  of  nonmodel-based  estimates  of  the  expected 
proportion  correct  (conditional  on  proficiency)  versus  the  proportion  correct  predicted  by  the  estimated 
item  response  function  (see  Chapter  9,  and  Mislevy  & Sheehan,  1987,  p.  302).  In  making  decisions  about 
excluding  items  from  the  final  scales,  a balance  was  sought  between  being  too  stringent,  hence  deleting 
too  many  items  and  possibly  damaging  the  content  representativeness  of  the  pool  of  scaled  items,  and 
being  too  lenient,  hence  including  items  with  model  fit  poor  enough  to  endanger  the  types  of  model- 
based  inferences  made  from  NAF.P  results.  A certain  degree  of  misfit  was  tolerated  for  a number  of  items 
included  in  the  final  scales. 

. Most  of  the  items  fit  the  model  well.  Items  excluded  from  the  analysis  of  the  1996  assessment 
were  the  same  items  that  were  deleted  from  the  1994  reading  long-term  trend  analysis.  Table  14-6  lists 
items  that  were  excluded  from  the  analysis  of  the  1996  long-term  trend  assessment. 

The  adequacy  of  the  assumption  of  a common  item  response  function  across  assessment  years 
was  also  evaluated  by  comparing  the  nonmodel-based  expected  proportions  for  each  assessment  year  to 
the  single,  model-based  item  response  function  fit  by  BILOG/PARSCALE.  Items  that  showed  clear 
evidence  of  functioning  differently  across  assessments  were  treated  as  separate  items  for  each  assessment 
year— that  is,  separate  item  response  functions  were  estimated  for  each  assessment.  As  was  the  case  with 
deleting  items,  in  making  decisions  about  scaling  items  separately  by  assessment  year,  a balance  was 
sought  between  being  too  stringent,  hence  splitting  too  many  items  and  possibly  damaging  the  common 
item  link  between  the  assessment  years,  and  being  too  lenient,  hence  including  items  with  model  fit  poor 
enough  to  endanger  the  model-based  trend  inferences.  These  separately  scaled  items  will  be  reexamined 
in  future  long-term  trend  assessments. 

At  age  9,  one  constructed-response  item  was  calibrated  separately  for  each  assessment  year. 
Examination  of  residual  plots  identified  the  item  as  functioning  differently  across  assessments.  Figure 
14-1  shows  item  N014502  from  the  analysis  for  grade  4/age  9.  Data  are  presented  for  1994  (squares),  and 
• for  1996  (asterisks)^  For  middle  proficiency  values,  the  two  sets  of  symbols  diverge,  and  the  discrepancy 
of  the  item  characteristic  curves  of  the  two  years  is  substantial.  The  top  (1994  data),  and  the  bottom 
(1996  data)  of  Figure  14-2  show  the  plots  for  the  item  treated  separately  by  assessment  year.  The  1996 
data  showed  poorer  fit.  In  order  to  maintain  the  link  for  the  trend,  this  item  was  kept  in  the  analysis  but 
with  the  1994  data  calibrated  separately  and  the  1996  data  excluded  from  the  final  calibration.  The 
remaining  misfit  is  relatively  small.  Overall,  one  long-term  trend  reading  item  was  calibrated  separately 
by  assessment  year.  Table  14-7  lists  the  item  that  was  calibrated  separately  across  assessment  years. 


^ The  size  of  the  symbols  are  proportional  to  the  estimated  number  of  students  at  a particular  scale  spore  level.  The  symbols  are 
ordinarily  larger  in  the  middle  of  the  theta  scale,  where  most  students’  scale  scores  fall. 
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Figure  14-1 

Example  Long-Term  Trend  Item  (NO 14502,  Age  9) 

Demonstrating  Differential  Item  Functioning  Across  Assessment  Years  1994  and  1996^ 


'This  plot  compares  empirical  and  model-based  estimates  of  the  item  response  function  (IRF).  The  smooth  curve 
represents  the  model-based  estimate  at  each  provisional  proficiency  level.  The  squares  represent  1994  data; 
asterisks  represent  1996  data. 

Note:  When  the  number  of  alternatives  of  a constructed-response  item  equaled  zero,  the  item  was  scored  in  only 
two  categories. 
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Figure  14-2 

Example  Long-Term  Trend  Item  (N014502,  Age  9)  ^ 

Fitting  Separate  Item  Response  Functions  for  Each  Assessment  Year 


THETA 


'The  plot  compares  empirical  and  model-based  estimates  of  the  item  response  function  (IRF).  The  smooth  curve 
represents  the  model-based  estimate  at  each  provisional  proficiency  level.  The  squares  represent  1994  data; 
asterisks  represent  1996  data. 


Note:  When  the  number  of  alternatives  of  a constnicted-response  item  equaled  zero,  the  item  was  scored  in  only 
two  categories. 
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Table  14-7 

Item  Calibrated  Separately  by  Assessment  Year  in  the  Reading  Long-Term  Trend  Analysis 


Age  Block Item Reason  for  Separate  Calibration 

9 B22  N014502  Poor  fit  across  assessments  to  common  item  response  function 


At  age  17,  two  items  (N002201  and  N002202)  caused  difficulty  in  scaling.  In  preliminary 
calibrations,  both  items  did  not  fit  the  model  well  and  had  large  slope-parameter  values  (3.9  and  5.0, 
respectively).  The  item  response  function  of  N002202  also  demonstrated  an  elevated  tail.  Further 
examination  of  the  items  indicated  that  this  might  be  due  to  local  dependence  of  the  items,  although 
neither  item  had  been  problematic  at  this  age  group  in  the  1994  assessment.  The  approach  of  fixing  the 
slope-parameter  was  taken  to  obtain  stable  item  parameter  estimates.  At  calibration  stage-two,  after  the 
estimation  of  the  proficiency  distribution  was  constrained  to  be  normally  distributed  and  calibrated  to 
convergence,  the  slope-parameter  of  N002201  was  fixed  at  the  value,  and  all  items  were  calibrated  to 
convergence.  Parameters  estimates  from  this  run  served  as  the  final  estimates  for  age  17. 

A list  of  the  items  scaled  for  each  of  the  ages,  along  with  their  item  parameter  estimates,  appears 
in  Appendix  D. 


14.5  GENERATION  OF  PLAUSIBLE  VALUES 

The  generation  of  plausible  values  was  conducted  independently  for  each  age/grade  level  for 
each  of  the  assessment  years.  The  item  parameters  from  BILOG/PARSCALE,  final  student  weights,  item 
responses,  and  selected  background  variables  were  used  with  the  computer  program  BGROUP  (described 
in  Chapter  11)  to  generate  the  values  for  each  age.  The  background  variables  included  student 
demographic  characteristics  (i.e.,  race/ethnicity  of  the  student,  highest  level  of  education  attained  by 
parents),  students’  perceptions  about  reading,  and  student  behavior  both  in  and  out  of  school  (e.g., 
amount  of  television  watched  daily,  amount  of  homework  done  each  day).  Appendix  C gives  the  codings 
for  the  conditioning  variables  for  the  three  age  groups.  Table  14-8  contains  a list  of  the  number  of 
background  contrasts  included  in  conditioning,  as  well  as  the  proportion  of  variance  accounted  for  by  the 
conditioning  model  for  each  age/grade. 


Table  14-8 

Proportion  of  Proficiency  Variance  Accounted  for  by  the  Conditioning  Model 
for  the  Reading  Long-Term  Trend  Assessment 


Age/Grade 

Number  of  Conditioning 
Contrasts’ 

Proportion  of  Proficiency 
Variance 

9/4 

49 

.32 

13/8 

49 

.35 

17/11 

47 

.34 

' Excluding  the  constant  term. 
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14.6  THE  FINAL  READING  LONG-TERM  TREND  SCALE 

The  linear  indeterminacy  of  the  long-term  trend  scale  was  resolved  by  linking  the  1996  long-term 
trend  scales  to  previous  long-term  trend  scales.  For  each  age,  the  item  parameters  from  the  joint 
calibration  based  on  data  from  both  1994  and  1996  were  used  with  the  1994  data  to  reestimate  plausible 
values  for  the  1994  data.  The  mean  and  standard  deviation  of  the  new  1994  estimates  were  calculated  and 
matched  to  the  mean  and  standard  deviation  of  the  old  1994  plausible  values  that  were  reported 
previously.  The  linear  constants  of  this  transformation  were  then  applied  to  transform  the  1996  scales  to 
the  1994  proficiency  metric.  The  transformation  equations  (described  in  Chapter  9)  that  resulted  from 
this  matching  of  the  first  two  moments  for  the  1994  data  are 

Age  9i  ©target  ~ 54.57  • ©calibrated  206.13, 

Age  13:  ©target  = 39.06  • ©calibrated  + 257.88,  and 

Age  17:  ©target  = 44.14  • ©caUbrated  + 283.19, 

where  ©target  denotes  values  on  the  final  transformed  scale,  and  ©calibrated  denotes  values  on  the  calibration 
scale.  Overall  summary  statistics  for  the  long-term  trend  samples  are  given  in  Table  14-9. 

As  in  the  past,  interpretation  of  the  long-term  trend  results  was  facilitated  through  the  provision 
of  scale  anchoring  information.  In  1984,  five  NAEP  reading  scale  levels  were  selected  as  anchor  points. 
These  points  (described  in  Trends  in  Academic  Progress)  are: 

150  = simple,  discrete  reading  tasks; 

200  = partially  developed  skills  and  understanding; 

250  = interrelation  of  ideas  and  generalizations; 

300  = understanding  complicated  information;  and 

350  = learning  from  specialized  reading  materials. 

Detailed  descriptions  of  the  skills  required  to  read  at  each  level  were  derived  and  benchmark 
exercises  were  selected  to  exemplify  each  level.  These  same  anchor  points  were  used  in  the  1988,  1990, 
1992,  1994,  and  1996  reading  long-term  trend  reports.  The  estimated  proportion  of  students  in  each 
reporting  category  who  are  at  or  above  each  anchor  point  was  examined  in  Trends  in  Academic  Progress. 

14.7  PARTITIONING  OF  THE  ESTIMATION  ERROR  VARIANCE 

For  each  age,  the  error  variance  of  the  final,  transformed  proficiency  mean  was  partitioned  into 
two  parts  as  described  in  Chapter  11.  This  analysis  yielded  estimates  of  the  proportion  of  error  variance 
due  to  sampling  students,  and  the  proportion  of  error  variance  due  to  the  latent  nature  of  ©.  These 
estimates  are  given  in  Table  14-10  (for  stability,  the  estimates  of  the  between-imputation  variance,  B,  in 
Equation  11.9  are  based  on  100  imputations).  More  detailed  information  is  available  for  gender  and 
race/ethnicity  subgroups  in  Appendix  E. 
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Table  14-9 

Means  and  Standard  Deviations  on  the  Reading  Long-Term  Trend  Scale 


Age 

Assessment 

Year 

All  Five  Plausible  Values 
Mean  S.D. 

9 

1984 

211.0 

41.1 

1988 

211.8 

41.2 

1990 

209.2 

44.7 

1992 

210.5 

40.4 

1994 

211.0 

40.5 

1996 

212.4 

40.5 

13 

1984 

257.1 

35.5 

1988 

257.5 

34.7 

1990 

256.8 

36.0 

1992 

259.8 

39.4 

1994 

257.9 

39.8 

1996 

259.1 

38.4 

17 

1984 

288.8 

40.3 

1988 

290.1 

37.1 

1990 

290.2 

41.3 

1992 

289.7 

43.0 

1994 

288.1 

44.4 

1996 

286.9 

42.3 

Table  14-10 

Estimation  Error  Variance  and  Related  Coefficients 
for  the  Reading  Long-Term  Trend  Assessment 

Proportion  of  Variance  Due  to  ... 

Total  Estimation  of 

Age 

Error  Variance 

Student  Sampling  Latency  of  6 

9 

1.01 

0.86 

0.14 

13 

0.93 

0.85 

0.15 

17 

1.08 

0.82 

0.18 
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Chapter  15 

DATA  ANALYSIS  FOR  THE  LONG-TERM  TREND 
MATHEMATICS  ASSESSMENT^ 

Jiahe  Qian  and  Norma  A.  Norris 
Educational  Testing  Service 


15.1  INTRODUCTION 

This  chapter  describes  the  analyses  performed  on  the  responses  to  the  cognitive  and  background 
items  in  the  1996  long-term  trend  assessment  of  mathematics.  The  emphasis  of  this  chapter  is  on  the 
methods  and  results  of  procedures  used  to  develop  the  IRT-based  scale  scores;  however,  some  attention 
is  given  to  the  analysis  of  constructed-response  items.  The  theoretical  underpinnings  of  the  IRT  and 
plausible  values  methodology  described  in  this  chapter  are  given  in  Chapter  11. 

The  objectives  of  the  mathematics  analyses  were  to  prepare  scale  values  and  perform  all  analyses 
necessary  to  produce  a long-term  trend  report  in  mathematics.  The  mathematics  long-term  trend  results 
include  the  years  1973,  1978, 1982, 1986,  1990,  1992, 1994,  and  1996.  The  results  of  1996  long-term 
trend  assessment  of  mathematics  are  presented  in  the  NAEP  1996  Trends  in  Academic  Progress: 
Achievement  ofU.S.  Students  in  Science,  1969  to  1996;  Mathematics,  1973  to  1996;  Reading,  1971  to 
1996;  and  Writing,  1984  to  1996  (Campbell,  Voelkl,  & Donahue,  1997). 

The  student  samples  that  were  administered  mathematics  items  in  the  1996  long-term  trend 
assessment  are  shown  in  Table  15-1.  See  Chapters  1 and  3 for  descriptions  of  the  target  populations  and 
the  sample  design  used  for  the  assessment. 


Table  15-1 

NAEP  1996  Mathematics  Long-Term  Trend  Student  Samples 


Sample 

Booklet  IDs 

Mode 

Cohort 

Assessed 

Time  of 
Testing 

Age 

Deflnition 

Modal 

Grade 

Number 

Assessed 

9 [MS-LTTrend] 

91-93 

Tape 

Age  9 

1/3/96  - 3/8/96  (Winter) 

CY 

4 

5,414 

13  [MS-LTTrend] 

91-93 

Tape 

Age  13 

10/9/95  - 12/22/95  (Fall) 

CY 

8 

5,658 

17  [MS-LTTrend] 

84-85 

Tape 

Age  17 

'3/11/96  - 5/10/96  (Spring) 

Not  CY 

11 

3,539 

LEGEND 


MS  Mathematics  and  science 

LTTrend  Long-term  trend  assessment:  booklets  are  identical  to  1986  long-term  trend  assessments 
Tape  Audiotape  administration 

CY  Calendar  year:  birthdates  (1996  sample)  in  1986  and  1982  for  ages  9 and  13 

Not  CY  Age  17  only  (1996  sample):  birthdates  between  October  1 and  September  30  of  the  appropriate  years 


^ Jiahe  Qian  was  the  primary  person  responsible  for  the  planning,  specification,  and  coordination  of  the  mathematics  long-term 
trend  analyses.  Computer  activities  for  all  long-term  trend  mathematics  scaling  and  data  analyses  were  performed  by  Norma 
Norris.  Nancy  Allen  and  Eiji  Muraki  provided  consultation. 
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Data  from  the  1996  long-term  trend  samples  that  contributed  to  the  trends  in  mathematics  achievement 
were  scaled  separately  from  the  1996  mathematics  main  samples.  Accordingly,  the  long-term  trend 
analysis  and  main  analysis  are  presented  in  separate  chapters.  This  chapter  pertains  to  the  scaling  of  the 
long-term  trend  data;  information  about  the  scaling  of  the  data  from  the  mathematics  main  assessment 
samples  is  presented  in  Chapter  12. 

The  long-term  trend  results  reported  in  the  NAEP  1996  Trends  in  Academic  Progress  are  based 
on  paced-tape  administrations  and  occur  at  all  age  levels.  The  samples  involved  in  the  analysis  are  shown 
as  9[MS-LTTrend],  13[MS-LTTrend],  and  17[MS-LTTrend]  in  Table  15-1.  For  ages  9 and  13,  the  long- 
term trend  booklets  contained  blocks  of  reading,  mathematics  and  science  items.  In  the  assessments,  the 
mathematics  and  science  blocks  were  administered  by  audiotape  to  pace  the  students  through  blocks  (the 
reading  blocks  were  only  presented  in  print  form).  The  age  17  long-term  trend  booklets  contained  only 
mathematics  and  science  blocks,  both  administered  by  paced-tape  recordings  as  well.  All  students 
received  a block  of  common  background  questions,  yet  distinct  for  each  age.  Subject-area  background 
questions  were  presented  in  the  cognitive  blocks.  The  booklets  for  the  age  9 and  age  13  samples 
(Booklets  91-93)  are  the  same  as  those  used  for  long-term  trend  assessments  in  1986,  1988,  1990,  1992, 
and  1994.  The  booklets  for  the  age  17  sample  (Booklets  84-85)  are  the  same  as  those  used  for  the  1986, 
1990,  1992,  and  1994  long-term  trend  assessments.  The  booklets  and  the  blocks  within  those  booklets  are 
listed  in  Tables  4-20  through  4-22  in  chapter  4. 

Table  15-2  clarifies  the  relationships  among  the  1996  long-term  trend  samples  and  samples  from 
previous  years.  For  ages  9,  13,  and  17,  the  paced-tape  bridge  to  the  1986  samples  allows  direct 
comparisons  between  the  samples  from  the  long-term  assessments  after  1990  and  the  1986  long-term 
trend  samples.  There  was  also  a paced-tape  administration  in  1988,  at  ages  9 and  13,  that  was  comparable 
to  the  other  years.  However,  a paced-tape  administration  was  not  conducted  at  age  17  in  1988.  Instead,  a 
noncomparable  paper-based  assessment  was  administered.  Hence,  1988  is  not  included  as  a point  in  the 
long-term  trend  reporting.  In  1986,  the  mathematics  long-term  trend  items  were  scaled  with  common 
items  from  the  1978  and  1982  assessments.  Because  the  1973  assessment  had  few  items  in  common  with 
the  current  assessment,  data  from  that  assessment  was  not  scaled  using  the  IRT  model  but  was  linked  to 
the  trend  line  by  a linear  transformation  involving  the  mean  proportion  correct  for  common  items  (See 
Expanding  the  New  Design:  The  NAEP  1985-86  Technical  Report  (Beaton,  1988)).  When  comparisons 
were  made  including  the  1970  and  1973  assessment  results,  z-tests  rather  than  t-tests  were  used  to  test 
statistical  significance  (See  Section  18.5.1).  The  1996  long-term  trend  assessment  was  linked  to  the 
1973,  1978,  and  1982  assessments  through  the  1986  assessment.  Information  about  previous  assessment 
years  is  available  in  Expanding  the  New  Design:  The  NAEP  1985-86  Technical  Report  (Beaton,  1988), 
The  NAEP  1992  Technical  Report  (Johnson  & Carlson,  1994),  and  The  NAEP  1994  Technical  Report  ’ 
(Alien,  Kline,  & Zelenak,  1996). 

Table  15-3  indicates  the  number  of  items  in  common  across  different  age  combinations.  Table 
15-4  shows  the  number  of  items  scaled  in  1996  that  were  common  across  assessment  years.  The  1986, 
1990,  1992,  1994,  and  1996  assessments  had  all  items  in  common.  For  age  9,  the  number  of  items 
common  across  assessment  years  1978  to  1996  was  only  35;  for  age  13,  it  was  56;  and  for  age  17,  it  was 
54.  Item  parameters  were  estimated  assuming  a univariate  scale,  since  the  number  of  items  presented  to 
each  student  was  small  and  there  were  too  few  items  to  estimate  several  content  area  scales  separately. 
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Table  15-2 

NAEP  Mathematics  Samples  Contributing  to  1996  Long-Term  Trend  Results,  1973-1996 


Cohort 

Time  of 

Mode  of 

Age 

Modal 

Assessed 

Year 

Sample 

Subjects 

Testing 

Administration 

DeHnition 

Grade 

Age  9 

1973 

Main 

MS 

Winter 

Tape 

CY 

4 

1978 

Main 

M 

Winter 

Tape 

CY 

4 

1982 

Main 

MCS 

Winter 

Tape 

CY 

4 

1986 

LTTrend^ 

MS 

Winter 

Tape^ 

CY 

4 

1990 

LTTrend' 

MS 

Winter 

Tape^ 

CY 

4 

1992 

LTTrend^ 

MS 

Winter 

Tape^ 

CY 

4 • 

1994 

LTTrend^ 

MS 

Winter 

Tape^ 

CY 

4 

1996 

LTTrend^ 

MS 

Winter 

Tape^ 

CY 

4 

Age  13 

1973 

Main 

MS 

Fall 

Tape 

CY 

8 

1978. 

Main 

M 

Fall 

Tape 

CY 

8 

1982 

Main 

MCS 

Fall 

Tape 

CY 

8 

1986 

LTTrend^ 

MS 

Fall 

Tape^ 

CY 

8 

1990 

LTTrend^ 

MS 

Fall 

Tape^ 

CY 

8 

1992 

LTTrend^ 

MS 

Fall 

Tape^ 

CY 

8 

1994 

LTTrend^ 

MS 

Fall 

Tape^ 

CY 

8 

1996 

LTTrend^ 

MS 

Fall 

Tape^ 

CY 

8 

Age  17 

1973 

Main 

MS 

Spring 

Tape 

NotCY 

11 

1978 

Main 

M 

Spring 

Tape 

NotCY 

11 

1982 

Main 

MCS 

Spring 

Tape 

Not  CY 

11 

1986 

LTTrend^ 

MS 

Spring 

Tape 

NotCY 

11 

1990 

LTTrend^ 

MS 

Spring 

Tape 

Not  CY 

11 

1992 

LTTrend^ 

MS 

Spring 

Tape 

Not  CY 

11 

1994 

LTTrend^ 

MS 

Spring 

Tape 

NotCY 

11 

1996 

LTTrend^ 

MS 

Spring 

Tape 

Not  CY 

11 

* Within  an  age  group,  these  samples  received  common  booklets. 

^ Mathematics  and  science  administered  by  audiotape,  reading  administered  by  print. 


LEGEND 


M 

Mathematics 

Main 

Main  assessment 

MS 

Mathematics  and  science 

Tape 

Audiotape  administration 

MCS 

Mathematics,  civics,  and  science 

CY 

Calendar  year:  birthdates  (1996  sample)  in 

LTTrend 

Long-term  trend:  booklets  are 
identical  to  the  long-term  trend 
assessment  of  1986. 

Not  CY 

1986  and  1982  for  ages  9 and  13 
Age  17  only  (1996  sample):  birthdates 
between  October  1 and  September  30  of  the 
appropriate  years 

The  steps  in  the  mathematics  long-term  trend  analysis  are  documented  in  the  following  sections. 
Consistent  with  the  procedures  in  earlier  NAEP  analyses,  the  first  step  was  to  calculate  standard  item 
statistics.  The  results  served  as  a check  for  data  entry  errors  and  as  a reasonableness  check  against  results 
from  previous  assessments. 

The  second  step  was  to  fit  an  IRT  model  to  the  data  from  the  1996  and  1994  assessments  for  each 
age  separately.  This  procedure  puts  item  parameters  and  ability  estimates  on  the  same  scale  across  years. 
The  same  item  may  have  different  item  parameters  for  different  age  groups. 
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Table  15-3 

Numbers  of  Scaled  Mathematics  Long-Term  Trend 
Items  Common  Across  Ages 


Age 

Booklet  Numbers 

Number  of  Items 

Total 

153 

9 only 

91-93 

32 

13  only 

91-93 

30 

17  only 

84-85 

41 

9 and  13  only 

91-93,  91-93 

20 

9 and  17  only 

91-93,  84-85 

0 

13  and  17  only 

91-93,  84-85 

27 

9,  13,  and  17 

91-93,91-93,  84-85 

3 

Table  15-4 

Numbers  of  Scaled  Mathematics  Long-Term  Trend  Items 
Common  Across  Assessments 


Assessment  Year 

1986, 1990,  1992,  1994,  1996 
1982,  1986,  1990,  1992,  1994,  1996 
1978,  1986,  1990,  1992,  1994,  1996 
1978,  1982,  1986,  1990,  1992,  1994,  1996 


Number  of  Items 
Age  9 Age  13  Age  17 


55 

80 

71 

53 

79 

65 

35 

56 

54 

35 

56 

54 

Next,  the  analysis  for  an  age  group  was  completed  by  the  creation  of  plausible  values  through  a 
multiple  imputation  estimation  procedure  in  which  item  parameter  estimates,  student  responses,  and 
student  background  information  were  combined  to  produce  the  most  precise  possible  estimates  of  student 
subgroup  ability.  Plausible  values  were  used  to  calculate  proficiency  means  for  the  entire  sample  and  for 
the  selected  subgroups. 

Finally,  the  scales  of  the  1996  trend  assessment  were  transformed  to  proficiency  scale  used  in 
previous  mathematics  trend  assessments.  These  proficiency  means  constitute  the  last  point  in  the 
mathematics  long-term  trend  from  1973  to  1996.  The  only  available  estimates  of  the  proficiency  means 
for  1 973  were  linked  via  extrapolation  to  the  IRT  scale,  but  the  data  from  that  year  was  never  scaled 
using  an  IRT  model. 


15.2  ITEM  ANALYSIS  FOR  THE  MATHEMATICS  LONG-TERM  TREND 
ASSESSMENT 

No  problems  in  coding,  formats,  or  data  were  detected.  The  conventional  item  analysis,  with 
results  displayed  in  Table  15-5,  was  performed  at  the  block  level  on  the  paced-tape  long-term  trend  data. 

Table  15-5  contains  the  number  of  items,  size  of  the  sample  administered  to  the  block,  mean 
weighted  proportion  correct,  mean  weighted  r-biserial,  and  mean  weighted  alpha  as  a measure  of 
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reliability  for  each  block.  The  average  values  were  calculated  using  examinee  weights  and  the  items  in 
the  block  that  were  scaled.  The  1996  item-level  statistics  were  not  very  different  from  those  for  the  1994 
assessment.  Similar  statistics  for  the  1994  assessment  were  reported  in  Table  16-5  of  The  NAEP  1994 
Technical  Report. 


Table  15-5 

Descriptive  Statistics  for  Item  Blocks  in  the 
Mathematics  Long-Term  Trend  Samples  (1996) 


Statistic 

Age  9 

Number  of  scaled  items 

Number  of  scaled  constructed  response  items 

Unweighted  sample  size 

Average  weighted  proportion  correct 

Average  weighted  r-biserial 

Weighted  alpha  reliability 


Ml 

Block 

M2 

M3' 

24 

26 

5 

9 

9 

0 

1,852 

1,841 

1,721 

.62 

.64 

.68 

.61 

.65 

.83 

.82 

.86 

.50 

Age  13 

Number  of  scaled  items 

Number  of  scaled  constructed  response  items 

Unweighted  sample  size 

Average  weighted  proportion  correct 

Average  weighted  r-biserial 

Weighted  alpha  reliability ^ 


36 

36 

8 

9 

8 

, 0 

1,928 

1,864 

1,866 

.68 

.62 

.66 

.59 

.55 

.68 

.87 

.85 

.61 

Age  17 

Number  of  scaled  items 

Number  of  scaled  constructed  response  items 

Unweighted  sample  size 

Average  weighted  proportion  correct 

Average  weighted  r-biserial 

Weighted  alpha  reliability 


33 

33 

5 

10 

5 

1 

1,848 

1,848 

1,691 

.66 

.67 

.56 

.69 

.63 

.76 

.90 

.87 

.54 

* This  block  is  mostly  calculator  items,  which  were  not  analyzed.  For  the  item  analysis,  students  who 
did  not  respond  to  any  items  in  the  block  were  omitted;  however,  such  students  were  assigned 
proficiencies  in  the  final  database. 


In  the  1996  mathematics  long-term  trend  assessment,  20  percent  of  the  samples  of  the 
constructed-response  items  were  used  to  check  the  interrater  reliability— the  score  agreement  between 
first  and  second  raters.  The  percent  of  exact  agreement  ranged  from  96.3  to  100  percent;  and  the 
intraclass  correlation  ranged  from  .902  to  1.00— except  .886  for  item  N269201  in  the  age  13  sample.  In 
general,  the  interrater  reliability  was  very  high  in  the  1996  mathematics  long-term  trend  assessment. 

The  correspondence  between  blocks,  booklets,  and  samples  is  given  for  the  mathematics  long- 
term trend  assessment  in  Tables  4-20  through  4-22  in  Chapter  4.  Common  labeling  of  these  blocks  across 
ages  does  not  denote  common  items. 
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15.3  IRT  SCALING  FOR  THE  MATHEMATICS  LONG-TERM  TREND 
ASSESSMENT 

15.3.1  Item  Parameter  Estimation 

The  scaling  process  began  with  the  estimation  of  item  parameters.  IRT  parameters  were 
estimated  using  the  NAEP  version  of  the  BILOG/PARSCALE  program  (Mislevy  & Bock,  1982;  Muraki 
& Bock,  1991)  described  in  Chapter  1 1 . Item  calibration  was  performed  separately  for  each  of  the  three 
age  groups,  using  the  total  combined  data  from  the  1994  and  1996  assessments.  Including  the  1994 
assessment  data  assures  that  item  parameters  will  be  similar  for  adjacent  assessments  so  that  year-to-year 
trends  will  not  be  distorted  by  abrupt  changes  in  calibration.  The  calibration  was  performed  on  the  entire 
sample  of  students,  resulting  in  a range  of  about  1,700  to  1,900  examinee  responses  to  each  item  in  each 
assessment  year.  The  calibration  was  based  on  student  weights  that  were  rescaled  for  the  1996  data  so 
that  the  sum  of  the  weights  equaled  the  unweighted  sample  size.  Also,  weights  for  the  1996  data  were 
restandardized  to  give  equal  weight  to  the  two  assessment  years  included  in  the  scaling.  As  with  the 
previous  assessment,  calculator  items  were  excluded  from  the  analysis.  Because  calculators  have  changed 
greatly  since  the  start  of  the  long-term  trend  assessment,  it  was  judged  that  calculator  questions  are  no 
longer  comparable  across  time.  These  items  were  kept  in  the  assessment,  since  excluding  them  would 
have  changed  the  testing  context. 

Since  parameters  for  items  in  blocks  Ml,  M2,  and  M3  were  estimated  separately  for  ages  9,  13, 
and  17,  items  administered  at  more  than  one  age  have  multiple  sets  of  item  parameter  estimates.  Items 
were  examined  for  lack  of  fit  with  the  data.  Those  that  exhibited  extreme  violation  of  IRT  assumptions 
(i.e.,  did  not  have  monotonically  increasing  item  characteristic  curves)  were  deleted  from  the  analysis,  as 
they  were  in  previous  assessments.  Other  items  were  deleted  because  they  were  calculator  items,  which 
were  not  considered  part  of  the  regular  assessment.  These  excluded  items  appear  in  Tables  15-6,  15-7, 
and  15-8.  As  a result  of  these  deletions,  55  items  were  scaled  for  age  9,  80  items  were  scaled  for  age  13, 
and  7 1 items  were  scaled  for  age  17.  Of  the  153  noncalculator  items  that  were  part  of  the  assessment, 
seven  items  (5%)  were  excluded  due  to  poor  fit  with  the  data.  A list  of  the  items  scaled  for  each  of  the 
ages,  along  with  their  item  parameter  estimates,  appears  in  Appendix  D. 


15.4  DERIVED  BACKGROUND  VARIABLES 

In  the  long-term  trend  analysis,  all  derived  background  variables  were  used  to  define  subgroups  of 
students  for  reporting.  For  this  reason,  these  variables  were  also  used  in  conditioning.  Derived  reporting 
variables  are  described  in  Appendix  B. 


Table  15-6 

Items  Deleted  from  the  Age  9 Mathematics  Long-Term  Trend  Analysis 


Booklet 

IDs 

Block 

Item 

Reason  for  Exclusion 

91 

Ml 

N252601 

Was  deleted  in  prior  assessment 

N262502 

Was  deleted  in  prior  assessment 

92 

M3 

N268221 

Calculator  item* 

N276021 

Calculator  item 

N276022 

Calculator  item 

N276821 

Calculator  item 

N276822 

Calculator  item 

N276823 

Calculator  item 

N277621 

Calculator  item 

N277622 

Calculator  item 

N277623 

Calculator  item 

N284021 

Calculator  item 

N284022 

Calculator  item 

* All  calculator  items  were  deleted  from  the  analysis. 


Table  15-7 

Items  Deleted  from  the  Age  13  Mathematics  Long-Term  Trend  Analysis 

Booklet 

IDs 

Block 

Item 

Reason  for  Exclusion 

91 

Ml 

N262502 

Was  deleted  in  prior  assessment 

93 

M2 

N261601 

Was  deleted  in  prior  assessment 

92 

M3 

N264521 

Calculator  item* 

N259921 

Calculator  item 

N276821 

Calculator  item 

N276822 

Calculator  item 

N276823 

Calculator  item 

N278921 

Calculator  item 

N278922 

Calculator  item 

N278923 

Calculator  item 

N278924 

Calculator  item 

N278925 

Calculator  item 

N280621 

Calculator  item 

N280622 

Calculator  item 

N280623 

Calculator  item 

N280624 

Calculator  item 

N280625 

Calculator  item 

N280626 

Calculator  item 

' All  calculator  items  were  deleted  from  the  analysis. 
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Table  15-8 


Items  Deleted  from  the  Age  17  Mathematics  Long-Term  Trend  Analysis 


Booklet 

IDs 

Block 

Item 

Reason  for  Exclusion 

84 

Ml 

N282801 

Was  deleted  in  prior  assessment 

N285701 

Was  deleted  in  prior  assessment 

84 

M2 

N266801 

Was  deleted  in  prior  assessment 

N255301 

Was  deleted  in  prior  assessment 

85 

M3 

N259921 

Calculator  item^ 

N264321 

Calculator  item 

N264521 

Calculator  item 

N267921 

Calculator  item 

N276821 

Calculator  item 

N276822 

Calculator  item 

N276823 

Calculator  item 

N278921 

Calculator  item 

N278922 

Calculator  item 

N278923 

Calculator  item 

N278924 

Calculator  item 

N278925 

Calculator  item 

N280621 

Calculator  item 

N280622 

Calculator  item 

N280623 

Calculator  item 

N280624 

Calculator  item 

N280625 

Calculator  item 

N280626 

Calculator  item 

N285321 

Calculator  item 

^All  calculator  items  were  deleted  from  the  analysis. 


15.5  GENERATION  OF  PLAUSIBLE  VALUES 

Plausible  values  were  calculated  separately  for  each  age  group.  In  this  phase  of  analysis,  student 
background  information  was  used  to  condition  item  responses  in  order  to  more  accurately  estimate 
average  subgroup  abilities.  The  conditioning  program  BGROUP  was  used  to  combine  NAEP 
BILOG/PARSCALE  item  parameters  with  weighted  item  responses  and  background  variables  to  produce 
posterior  ability  estimates  called  plausible  values.  As  defined  in  Chapter  11,  BGROUP  is  an  enhanced 
version  of  the  original  conditioning  program,  MGROUP.  Plausible  values  are  not  test  scores  in  the  usual 
sense,  but  can  be  used  to  provide  consistent  estimates  of  population  characteristics.  There  were  53 
contrasts  in  the  conditioning  model  at  age  9,  56  at  age  13,  and  63  at  age  17.  Appendix  C gives  the 
codings  for  the  conditioning  variables  for  the  three  age  groups.  A check  was  made  on  the  distributions  of 
the  plausible  values  for  each  age,  including  inspection  of  the  whole  group  and  subgroup  means  and 
standard  deviations.  Table  15-9  contains  a list  of  the  number  of  background  contrasts  included  in 
conditioning,  as  well  as  the  proportion  of  variance  accounted  for  by  the  conditioning  model  for  each 
age/grade. 
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Table  15-9 

Proportion  of  Proficiency  Variance  Accounted  for  by  the  Conditioning  Model 
for  the  Mathematics  Long-Term  Trend  Assessment 


Number  of 

Proportion  of 

Age/Grade 

Conditioning  Contrasts' 

Proficiency  Variance 

-,-P  — - 

9/4 

53 

.37 

13/8 

56 

.35 

17/12 

63 

.57 

‘ Including  the  constant  term. 


15.6  THE  FINAL  MATHEMATICS  LONG-TERM  TREND  SCALE 

Since  the  plausible  value  (theta)  scales  have  a linear  indeterininacy,  comparisons  with  previous 
assessments  will  be  sensible  only  if  the  scale  is  linearly  transformed  to  a meaningful  metric.  This 
indeterminacy  was  resolved  by  linking  the  1996  scales  to  previous  long-term  trend  scales.  The  1996  data 
had  to  be  transformed  to  compensate  for  linear  changes  in  the  scale  due  to  employing  newly  estimated 
item  parameters  and  new  BGROUP  conditioning  parameters  in  1996.  The  transformation  was 
accomplished  by  first  reestimating  the  1994  student  abilities  using  1996  item  parameters  and  1996 
BGROUP  parameters.  (For  score  metric  transformation,  see  Chapter  9.)  The  new  1994  ability  estimates 
were  then  equated  to  the  old  1994  ability  estimates  by  matching  the  first  two  moments  (i.e.,  the  mean  and 
standard  deviation).  The  constants  for  this  transformation  were  then  applied  to  the  1996  data.  The 
transformation  equations  that  resulted  are 

Age  9:  Gtarget  = 34.04  • Gcalibrated  + 230.46 

Age  13:  Gtarget  = 33.08  • Gcalibrated  + 273.91 

Age  17:  Gtarget  = 30.46  • Gcalibrated  + 306.57, 

where  Gtarget  denotes  an  individual’s  value  on  the  final  transformed  scale  of  the  1996  data  and  Gcalibrated 
denotes  an  individual’s  value  on  the  original  1996  theta  scale.  Overall  summary  statistics  for  the 
long-term  trend  samples  are  given  in  Table  15-10.  For  the  descriptions  of  the  results  of  the  mathematics 
long-term  trend  study,  see  NAEP  1996  Trends  in  Academic  Progress  (Campbell,  Voelkl,  & Donahue, 
1997). 

To  provide  a context  for  interpreting  the  overall  mathematics  long-term  trend  results;  the  NAEP 
mathematics  results  were  “anchored”  at  five  NAEP  mathematic  scale  levels.  These  points  (described  in 
the  NAEP  1996  Trends  in  Academic  Progress)  are: 

150  = simple  arithmetic  facts; 

200  = beginning  skills  and  understanding; 

250  = numerical  operations  and  beginning  problem  solving; 

300  = moderately  complex  procedures  and  reasoning;  and 

350  = multi-step  problem  solving  and  algebra. 

These  same  anchor  points  were  used  in  the  1978,  1982,  1986,  1990,  1992,  and  1994  mathematics  long- 
term  trend  reports. 
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Table  15-10 

Means  and  Standard  Deviations  on  the 
Mathematics  Long-Term  Trend  Proficiency  Scale 


Age 

Assessment 

All  Five  Plausible  Values 
Mean  S.  D. 

9 

1978 

218.6 

36.0 

1982 

219.0 

34.8 

1986 

221.7 

34.0 

1990 

229.6 

32.9 

1992 

229.6 

33.1 

1994 

231.1 

33.2 

1996 

231.0 

33.8 

13 

1978 

264.1 

39.0 

1982 

268.6 

33.4 

1986 

269.0 

30.8 

1990 

270.4 

31.3 

1992 

273.1 

30.9 

1994 

274.3 

32.4 

1996 

274.3 

31.6 

17 

1978 

300.4 

34.9 

1982 

298.5 

32.4 

1986 

302.0 

31.0 

1990 

304.6 

31.3 

1992 

306.7 

30.1 

1994 

306.2 

30.2 

1996 

307.2 

30.2 

15.7  PARTITIONING  OF  THE  ESTIMATION  ERROR  VARIANCE 

* scale,  the  error  variance  of  the  final  transformed  proficiency  mean  was  partitioned 
as  described  in  Chapter  11.  The  partition  of  error  variance  consists  of  two  parts:  the  proportion  of  error 
variance  due  to  sampling  students  (sampling  variance)  and  the  proportion  of  error  variance  due  to  the  fact 
that  proficiency,  0,  is  a latent  variable  that  is  estimated  rather  than  observed.  Table  15-11  contains 
estimates  of  the  total  error  variance,  the  proportion  of  error  variance  due  to  sampling  students  and  the 
proportion  of  error  variance  due  to  the  latent  nature  of  0 (for  stability,  the  estimates  of  the  between- 
imputation  variance,  B,  in  Equation  1 1.9  are  based  on  100  imputations.). 


Table  I5-II 

Estimation  Error  Variance  and  Related  Coefficients 
for  the  Mathematics  Long-Term  Trend  Assessment 


Age 

Total  Estimation  Error  Variance 

Proportion  of  Variance  Due  to  . . . 
Student  Sampline  Latency  of  0 

9 

0.66 

0.85 

0.15 

13 

0.66 

0.89 

0.11 

17 

1.24 

0.93 

0.07 

More  detailed  information  is  available  for  gender  and  race/ethnicity  subgroups  in  Appendix  E. 
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Chapter  16 


DATA  ANALYSIS  FOR  THE  LONG-TERM  TREND 
SCIENCE  ASSESSMENT^ 


Jinming  Zhang  and  Norma  A.  Norris 
Educational  Testing  Service 


16.1  INTRODUCTION 

This  chapter  describes  the  analyses  performed  on  the  responses  to  the  cognitive  and  background 
items  in  the  1996  long-term  trend  assessment  of  science.  The  objectives  of  the  science  analyses  are  to 
prepare  scale  values  and  perform  all  analyses  necessary  to  produce  a long-term  trend  report  in  science. 

The  results  obtained  from  these  analyses  includes  the  years  1969-1970,  1973,  1977,  1982,  1986,  1990, 
1992,  1994,  and  1996,  and  are  presented  in  the  NAEP  1996  Trends  in  Academic  Progress:  Achievement 
ofU.S.  Students  in  Science,  1969  to  1996;  Mathematics,  1973  to  1996;  Reading,  1971  to  1996;  and 
Writing,  1984  to  1996  (Campbell,  Voelkl,  & Donahue,  1997).  The  theoretical  underpinnings  of  the  IRT 
and  the  plausible  values  methodology  used  in  this  chapter  are  described  in  Chapter  9 and  Chapter  11,  and 
are  therefore  not  detailed  here. 

The  student  samples  that  were  administered  science  items  in  the  1996  long-term  trend  assessment 
are  shown  in  Table  16-1  as  9[MS-LTTrend],  13[MS-LTTrend],  and  17[MS-LTTrend].  (See  Chapters  1 
and  3 for  descriptions  of  the  target  populations  and  the  sample  design  used  for  the  assessment.)  Data 
from  the  long-term  trend  samples  that  contributed  to  the  trends  in  science  achievement  were  scaled 
separately  from  the  1996  science  main  focused-BIB  samples.  Accordingly,  the  long-term  trend  and  mam 
analyses  are  presented  in  separate  chapters.  Information  about  the  scaling  of  the  data  from  the  science 
main  focused-BIB  samples  is  presented  in  Chapter  13. 

The  science  long-term  trend  results  reported  in  the  NAEP  1996  Trends  in  Academic  Progress  are 
based  on  paced-tape  administrations  at  all  three  age  levels.  For  ages  9 and  13,  the  long-term  trend 
booklets  contain  one  mathematics  block,  one  reading  block,  and  one  science  block.  The  science  and 
mathematics  blocks  were  paced  by  tape-recordings  (i.e.,  tape-recordings  were  used  to  be  sure  that  the 
items  were  read  in  a consistent  manner  in  every  session  and  pace  students  through  the  blocks)  and  the 
reading  block  was  presented  in  print  form  only  and  were  not  paced  by  tape-recordings.  The  age  17  long- 
term trend  booklets  contain  only  mathematics  and  science  blocks,  both  paced  by  tape-recordings.  All 
students  received  a block  of  common  background  questions,  distinct  for  each  age.  Subject-area 
background  questions  were  presented  in  the  cognitive  blocks.  The  booklets  for  the  age  9 and  age  13 
samples  (Booklets  91-93)  and  the  booklets  for  the  age  17  samples  (Booklets  84-85)  are  the  same  as  those 
used  for  long-term  trend  assessments  in  1986,  1990,  1992,  and  1994.  The  booklets  and  the  blocks  within 
those  booklets  are  listed  in  Chapter  4.  Additional  information  about  all  of  the  items  in  these  blocks  is  also 
found  in  that  chapter.  This  chapter  includes  specific  information  about  the  long-term  trend  items  that 
were  scaled. 


* Jinming  Zhang  was  the  primary  person  responsible  for  the  planning,  specification,  and  coordination  of  the  science  long-term 
trend  analyses.  Computer  activities  for  all  long-term  trend  science  scaling  and  data  analyses  were  performed  by  Norma  Noms. 
Nancy  Allen,  Eiji  Muraki,  and  John  Donoghue  provided  consultation. 
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Table  16-1 


NAEP  1996  Long-Term  Trend  Science  Student  Samples 


Sample 

Booklet 

IDs 

Mode 

Cohort 

Assessed 

Time  of 
Testing 

Age 

Definition 

Modal 

Grade 

Number 

Assessed 

9 [MS-LTTrend] 

91-93 

Tape 

Age  9 

1/3/96  — 3/8/96  (Winter) 

CY 

4 

5,414 

13  [MS-LTTrend] 

91-93 

Tape 

Age  13 

10/9/95  - 12/22/95  (Fall) 

CY 

8 

5,658 

17  [MS-LTTrend] 

84-85 

Tape 

Age  17 

3/11/96-5/10/96  (Spring) 

NotCY 

11 

3,539 

LEGEND 


MS  Mathematics  and  science 

LTTrend  Long-term  trend  assessment:  booklets  are  identical  to  1986  long-term  trend  assessments 
Tape  Audiotape  administration 

CY  Calendar  year:  birthdates  in  1986  and  1982  for  ages  9 and  13,  respectively 

Not  CY  Age  17  only:  birthdates  between  October  1,  1978,  and  September  30,  1979 


Table  16-2  clarifies  the  relationships  among  the  1996  long-term  trend  samples  and  samples  from 
previous  years.  For  all  ages,  the  1996  science  long-term  trend  samples  allow  direct  comparisons  with 
1986,  1990,  1992,  and  1994  long-term  trend  samples  because  the  same  booklets  were  used  in  these 
assessments.  There  was  also  a tape  administration  in  1988  at  ages  9 and  13  that  was  comparable  to  the 
other  years.  However,  a tape  administration  was  not  conducted  at  age  17  in  1988.  Instead,  a 
noncomparable  paper-based  assessment  was  conducted.  Hence,  1988  is  not  included  as  a point  in  the 
long-term  trend  reporting.  In  1986,  the  science  long-term  trend  items  were  scaled  with  common  items 
from  the  1977  and  1982  assessments.  Because  of  the  small  number  of  items  in  common  with  those  in  the 
1969-70  and  1973  assessments,  data  from  the  1969-70  and  1973  assessments  were  not  scaled  using  the 
IRT  model,  but  were  linked  to  the  long-term  trend  line  by  a linear  transformation  involving  the  logit  of 
mean  proportion  correct  for  common  items.  When  comparisons  were  made  including  the  1969-70  and 
1973  assessment  results,  z-tests  rather  than  t-tests  were  used  to  test  statistical  significance  (See  Section 
18.5.1).  From  1990,  each  new  long-term  trend  assessment  was  linked  to  the  previous  assessments 
through  the  last  assessment.  For  instance,  the  1996  long-term  trend  assessment  was  linked  to  the  previous 
assessments  thrpugh  the  1994  long-term  trend  assessment.  Information  about  previous  assessment  years, 
including  1969-70  and  1973,  is  available  in  Chapter  1 1 of  Expanding  the  New  Design:  The  NAEP  1985- 
86  Technical  Report  (Yamamoto,  1988),  Chapter  14  of  The  NAEP  1990  Technical  Report  (Johnson  & 
Allen,  1992),  Chapter  14  of  The  NAEP  1992  Technical  Report  (Allen  & Isham,  1994),  and  Chapter  17  of 
The  NAEP  1994  Technical  Report  (Swinton,  Allen,  Isham  & Chen,  1996). 

The  numbers  of  scaled  items  in  common  across  different  ages  are  presented  in  Table  16-3.  As 
was  done  with  previous  long-term  trend  analyses,  each  age  was  scaled  separately  and  the  long-term  trend 
scales  are  univariate.  Derivation  of  scales  for  specific  content  areas  was  not  feasible  given  the  limited 
number  of  items  presented  to  students  in  the  long-term  trend  samples.  The  number  of  items  scaled  in 
1996  that  were  common  across  assessment  years  is  presented  in  Table  16-4. 

The  steps  in  the  science  long-term  trend  analysis  are  documented  in  the  following  sections.  As  is 
usual  in  NAEP  analyses,  the  first  step  was  to  gather  item-level  and  block-level  information.  Then,  the 
cognitive  items  were  calibrated  according  to  the  IRT  model.  Next,  derived  background  variables  were 
calculated,  and  plausible  values  were  generated  after  conditioning  on  available  background  variables  and 
selected  two-way  interactions.  Finally,  the  scale  values  were  placed  on  the  final  science  long-term  trend 
scale  used  in  previous  trend  assessments. 


V.' 


376 


Table  16-2 


NAEP  Science  Samples  Contributing  to  1996  Long-Term  Trend  Results,  1970-1996 


Cohort 

Time  of 

Mode  of 

Age 

Modal 

Assessed 

Year 

Sample 

Subjects 

Testing 

Administration 

Definition 

Grade 

Age  9 

1970 

Main 

SWC 

Winter 

Tape 

CY 

4 

1973 

Main 

MS 

Winter 

Tape 

CY 

4 

1977 

Main 

SCI 

Winter 

Tape 

CY 

4 

1982 

Main 

MSC 

Winter 

Tape 

CY 

4 

1986 

LTTrend' 

MS 

Winter 

Tape^ 

CY 

4 

1990 

LTTrend' 

MS 

Winter 

Tape^ 

CY 

4 

1992 

LTTrend' 

MS 

Winter 

Tape^ 

CY 

4 

1994 

LTTrend' 

MS 

Winter 

Tape^ 

CY 

4 

1996 

LTTrend' 

MS 

Winter 

Tape^ 

CY 

4 

Age  13 

1970 

Main 

SWC 

Fall 

Tape 

CY 

8 

1973 

Main 

MS 

Fall 

Tape 

CY 

8 

1977 

Main 

SCI 

Fall 

Tape 

CY 

8 

1982 

Main 

MSC 

Fall 

Tape 

CY 

8 

1986 

LTTrend' 

MS 

Fall 

Tape^ 

CY 

8 

1990 

LTTrend' 

MS 

Fall 

Tape^ 

CY 

8 

1992 

LTTrend' 

MS 

Fall 

Tape^ 

CY 

8 

1994 

LTTrend' 

MS 

Fall 

Tape^ 

CY 

8 

1996 

LTTrend' 

MS 

Fall 

Tape^ 

CY 

8 

Age  17 

1969 

Main 

SWC 

Spring 

Tape 

NotCY 

11 

1973 

Main 

MS 

Spring 

Tape 

NotCY 

11 

1977 

Main 

SCI 

Spring 

Tape 

Not  CY 

11 

1982 

Main 

MSC 

Spring 

Tape 

NotCY 

11 

1986 

LTTrend' 

MS 

Spring 

Tape 

NotCY 

11 

1990 

LTTrend' 

MS 

Spring 

Tape 

NotCY 

11 

1992 

LTTrend' 

MS 

Spring 

Tape 

NotCY 

11 

1994 

LTTrend' 

MS 

Spring 

Tape 

NotCY 

11 

1996 

LTTrend' 

MS 

Spring 

Tape 

Not  CY 

11 

‘ Within  an  age  group,  these  samples  received  common  booklets. 

^Mathematics  and  science  administered  by  audiotape,  reading  administered  by  print. 


LEGEND 

SCI  Science 

MS  Mathematics  and  science 

MSC  Mathematics,  science,  and  citizenship 

SWC  Science,  writing,  and  citizenship 

Main  Main  assessment 


Long-term  trend:  booklets  are  identical  to  the  long- 
term trend  assessment  of  1986 
Audiotape  administration 

Calendar  year:  birthdates  in  1986  and  1992  for  ages 
9 and  13  in  the  1996  assessment 
Age  17  only:  birthdates  between  October  1 and 
September  30  of  the  appropriate  years 


LTTrend 

Tape 

CY 

Not  CY 
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Table  16-3 

Numbers  of  Scaled  Science  Long-Term  Trend 
Hems  Common  Across  Ages 


Age 

Booklet  Numbers 

Number  of  Items 

9 only 

91-93 

55 

13  only 

91-93 

30 

17  only 

84-85 

32 

9 and  13  only 

91-93,91-93 

0 

9 and  17  only 

91-93,  84-85 

0 

13  and  17  only 

91-93,  84-85 

45' 

9,  13, and  17 

91-93,  91-93,  84-85 

1 

Total 

163 

One  of  these  items  (N406303)  was  treated  as  a different  item  from  1990  in  the 
scaling  of  the  1992  assessment,  but  only  for  age  13.  It  was  treated  as  an  item 
common  to  1992,  1994  and  1996  for  all  ages  in  the  1994  and  1996  assessments. 


Table  16-4 

Numbers  of  Scaled  Science  Long-Term  Trend 
Items  Common  Across  Assessments 


Number  of  Items 

Assessment  Years  Age  9 Age  13  Age  17 


1986,  1990,  1992, 1994,1996 
1982,  1986,  1990,  1992,  1994,1996 
1977,  1986,  1990,  1992,  1994,1996 
1977,  1982,  1986, 1990,  1992,  1994,  1996 


56 

76 

78 

10' 

58 

47 

56 

76 

76 

10' 

58^ 

45 

' Twenty-four  items  common  to  years  1977  and  1982,  but  not  later  years,  were  included  in  the  1986  scaling  of  these 
items  to  stabilize  the  estimation  of  the  item  parameters.  See  Expanding  the  New  Design:  The  NAEP  1985-86 
Technical  Report  for  more  information. 

^ One  of  these  items  (N406303)  was  treated  as  a different  item  from  1 990  in  the  scaling  of  the  1 992  assessment  but 

only  for  age  13.  It  was  treated  as  an  item  common  to  1992,  1994  and  1996  in  the  1994  and  1996  assessments  for  all 
ages. 


16.2  ITEM  ANALYSIS  FOR  THE  SCIENCE  LONG-TERM  TREND  ASSESSMENT 

Conventional  item  analyses  did  not  identify  any  difficulties  with  the  1996  long-term  trend  data 
for  the  1996  samples  that  bridge  to  1986.  Table  16-5  contains  information  about  the  science  long-term 
trend  blocks.  These  blocks  were  presented  to  samples  9[MS-LTTrend],  13[MS-LTTrend],  and  17[MS- 
LTTrend].  At  all  ages,  the  blocks  labeled  SI,  S2,  and  S3  were  presented  intact  to  students  in  the  1986, 
1990,  1992,  1994,  and  1996  long-term  trend  samples.  The  age  9 and  age  13  blocks  appeared  in  Booklets 
91  through  93.  For  age  17,  Block  S3  was  in  Booklet  84,  and  Blocks  SI  and  S2  were  in  Booklet  85.  The 
correspondence  between  blocks,  booklets,  and  samples  is  given  for  the  long-term  trend  assessment  in 

Tables  4-14  through  4-16  in  Chapter  4.  Common  labeling  of  these  blocks  across  ages  does  not  denote 
common  items. 


Table  16-5  contains  the  number  of  scaled  items,  size  of  the  sample  administered  to  the  block, 
mean  weighted  proportion  correct,  mean.weighted  r-biserial,  and  mean  weighted  alpha  as  a measure  of 
reliability  for  each  block.  The  average  values  were  calculated  using  examinee  sampling  weights  and  the 
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responses  to  the  items  in  the  block  that  were  scaled.  On  average,  the  1996  item-level  statistics  were  not 
very  different  from  those  for  the  1994  assessments.  The  percent  of  examinees  not  reaching  items  in  the 
science  long-term  trend  blocks  was  almost  always  zero  because  the  items  were  administered  with  a tape- 
recording to  pace  response  time. 

Table  16-5 

Descriptive  Statistics  for  Item  Blocks  in  the  Science  Long-Term  Trend  Samples  (1996) 

Blocks 

Statistic SI S2 S3 


Age  9 

Number  of  scaled  items 

Number  of  scaled  constructed-response  items 

Unweighted  sample  size 

Average  weighted  proportion  correct 

Average  weighted  r-biserial 

Weighted  alpha  reliability 

Age  13 

Number  of  scaled  items 

Number  of  scaled  constructed-response  items 

Unweighted  sample  size 

Average  weighted  proportion  correct 

Average  weighted  r-biserial 

Weighted  alpha  reliability 

Age  17 

Number  of  scaled  items 

Number  of  scaled  constructed-response  items 

Unweighted  sample  size 

Average  weighted  proportion  correct 

Average  weighted  r-biserial 

Weighted  alpha  reliability 


17 

20 

19 

0 

0 

0 

1,852 

1,721 

1,841 

0.62 

0.58 

0.71 

.0.57 

0.48 

0.58 

0.71 

0.64 

0.73 

23 

30 

23 

0 

0 

0 

1,928 

1,866 

1,864 

0.53 

0.56 

0.61 

0.53 

0.50 

0.52 

0.74 

0.79 

0.72 

24 

31 

23 

0 

0 

0 

1,691 

1,691 

1,848 

0.65 

, 0.65 

0.61 

0.49 

0.54 

0.64 

0.68 

0.79 

0.82 

16.3  IRT  SCALING  FOR  THE  SCIENCE  LONG-TERM  TREND  ASSESSMENT 
16.3.1  Item  Parameter  Estimation 

The  first  step  in  the  scaling  process  was  the  estimation  of  item  parameters  for  the  long-term  trend 
items.  This  item  calibration  was  performed  using  the  NAEP  version  of  the  BILOG/PARSCALE  program, 
which  combines  Mislevy  and  Bock’s  (1982)  BILOG  and  Muraki  and  Bock  s (1991)  PARSCALE 
computer  programs.  Items  were  calibrated  separately  for  each  of  the  three  age  groups,  using  combined 
data  from  the  1994  and  1996  assessment  years  and  treating  each  assessment  sample  as  a sample  from  a 
separate  subpopulation.  In  several  previous  long-term  trend  analyses,  combined  data  from  the  last 
assessment  and  the  current  assessment  were  used  for  item  parameter  estimation.  The  purposes  for 
including  the  last  long-term  trend  assessment  data  are  to  assure  that  item  parameter  estimates  will  be 
similar  for  adjacent  assessments  so  that  year-to-year  trends  will  not  be  distorted  by  abrupt  changes  in 
calibration,  and  to  make  it  possible  to  link  the  current  long-term  trend  assessment  to  the  previous 
assessments  through  the  last  assessment.  Student  weights  were  used  for  the  calibration.  To  ensure  that 
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each  assessment  year  had  a similar  influence  on  the  calibration,  student  weights  for  each  1994  age  group 
were  multiplied  by  a constant,  to  adjust  them  to  have  the  same  sum  as  the  sum  of  the  student  weights  for 
the  corresponding  1996  age  group. 


Although  other  items  were  examined  for  irregularities,  only  items  that  were  deleted  from  the 
previous  scaling  of  the  paced-tape  long-term  trend  data  were  excluded  in  the  1996  analysis.  Eight  percent 
of  the  items  (18  items)  administered  to  the  long-term  trend  sample  were  excluded  from  analyses  of 
previous  assessments.  The  deleted  items  appear  in  Table  16-6.  As  a result  of  these  deletions,  56  items 
were  scaled  for  age  9,  76  items  were  scaled  for  age  13,  and  78  items  were  scaled  for  age  17.  A list  of  the 
items  scaled  for  each  of  the  ages,  along  with  their  item  parameter  estimates,  appears  in  Appendix  D. 


Table  16-6 

Items  Deleted  from  the  Paced-Tape  Science  Long-Term  Trend  Analysis 


Booklet 


Age 

IDs 

Block 

Item 

9 

91 

SI 

N400201 

92 

S2 

N401701 

92 

S2 

N402003 

92 

S2 

N402004 

92 

S2 

N402601 

92 

S2 

N402603 

93 

S3 

N403802 

91 

SI 

N404902 

91 

SI 

N404903 

92 

S2 

N407501 

93 

S3 

N409401 

93 

S3 

N409402 

93 

S3 

N409403 

93 

S3 

N409801 

17 

85 

SI 

N4 10001 

85 

SI 

N4 10002 

85 

SI 

N4 10301 

85 

S2 

N407402 

Reason  for  Exclusion 

Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 

Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous assessment 

Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 
Excluded  in  previous  assessment 


16.3.2  Derived  Background  Variables 


In  the  long-term  trend  analysis,  all  variables  derived  from  background  questions  were  used  both 
in  generating  plausible  values  and  in  reporting  (to  define  subgroups).  Derived  conditioning  and  reporting 
variables  are  described  in  Appendix  B. 
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16.4  GENERATION  OF  PLAUSIBLE  VALUES 


The  generation  of  plausible  values  was  conducted  independently  for  each  age  group.  The  item 
parameters  from  NAEP-BILOG/PARSCALE,  final  student  weights,  item  responses  and  selected 
background  variables  (conditioning  variables)  were  used  with  the  computer  program  BGROUP 
(described  in  Chapter  1 1)  in  order  to  generate  the  plausible  values  for  each  student.  There  were  49 
contrasts  in  the  conditioning  model  (11.8)  at  age  9,  excluding  an  overall  constant,  52  at  age  13,  and  58  at 
age  17.  Appendix  C gives  the  codings  for  the  conditioning  variables  for  the  three  age  groups.  A check  on 
the  distributions  of  the  plausible  values  for  each  age  was  made.  The  generation  of  plausible  values  is 
described  in  more  detail  in  Chapters  9 and  11.  Table  16-7  contains  a list  of  the  number  of  background 
contrasts  included  in  conditioning,  as  well  as  the  proportion  of  variance  accounted  for  by  the 
conditioning  model  for  each  age.  This  proportion  is  the  ratio  of  the  difference  between  the  total  variance 
and  the  BGROUP  residual  variance,  divided  by  the  total  variance.  The  total  variance  is  the  mean  of  the 
five  theta-scale  variances  obtained  by  their  respective  plausible  values. 


Table  16-7 

Proportion  of  Proficiericy  Variance  Accounted  for  by  the  Conditioning  Model 
for  the  Science  Long-Term  Trend  Assessment 


Age 

Number  of 

Conditioning  Contrasts' 

Proportion  of 
Proficiency  Variance 

9 

49 

0.33 

13 

52 

0.37 

17 

58 

0.46 

* Excluding  the  constant  term. 


16.5  THE  FINAL  SCIENCE  LONG-TERM  TREND  SCALE 

The  linear  indeterminacy  of  the  long-term  trend  scale  was  resolved  by  linking  the  1996  long-term 
trend  scales  to  the  previous  long-term  trend  scales  using  the  following  procedure.  For  each  age,  the  item 
parameters  based  on  combined  data  from  1994  and  1996  were  used  with  the  1994  data  to  find  plausible 
values  for  the  1994  data.  The  mean  and  standard  deviation  of  all  of  the  plausible  values  (theta  scale)  were 
calculated  and  matched  to  the  mean  and  standard  deviation  of  all  of  the  science  long-term  trend  scale 
scores  (final  reporting  scale)  based  on  the  1994  item  parameters  and  1994  data  as  reported  in  the  NAEP 
1994  Technical  Report.  The  transformations  that  resulted  from  this  matching  of  the  first  two  moments  for 
the  1994  data  are 


Age  9:  Gtarget  = 38.57  • Gcalibrated  + 232.56, 

Age  13:  Gtarge.  = 40.1 1 • Gcaiibrated  + 255.54,  and 
Age  17:  Gtarget  ~ 48.28  • Gcaiibrated  293.82, 


where  Gtarget  denotes  values  on  the  final  reporting  scale  of  the  1996  data  and  Gcaiibrated  denotes  values  on  the 
original  1996  calibration  (theta)  scale.  Overall  summary  statistics  for  the  long-term  trend  scales  are  given 
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in  Table  16-8.  The  detailed  science  long-term  trend  results  from  the  analyses  described  in  this  chapter  are 
reported  in  NAEP  1996  Trends  in  Academic  Progress. 


Table  16-8 

Means  and  Standard  Deviations  on  the  Science  Long-Term  Trend  Scale 


_ Age 

Assessment 

AH  Five  Plausible  Values 
Mean  S.  D. 

9 

1977 

219.9 

44.9 

1982 

220.8 

40.9 

1986 

224.3 

41.6 

1990 

228.7 

40.2 

1992 

230.6 

39.9 

1994 

231.0 

40.9 

1996 

229.7 

42.2 

13 

1977 

247.4 

43.5 

1982 

250.1 

38.6 

1986 

251.4 

36.6 

1990 

255.2 

37.6 

1992 

258.0 

36.9 

1994 

256.8 

37.2 

1996 

256.0 

38.4 

17 

1977 

289.5 

45.0 

1982 

283.3 

46.7 

1986 

288.5 

44.4 

1990 

290.4 

46.2 

1992 

294.1 

44.7 

1994 

294.0 

45.6 

1996 

295.7 

45.1 

As  in  the  past,  interpretation  of  the  science  long-term  trend  results  was  facilitated  through  the 
provision  of  scale  anchoring  information.  In  1986,  five  science  scale  levels  were  selected  as  anchor 
points,  using  the  process  described  in  Expanding  the  New  Design:  The  1985-86  Technical  Report 
(Beaton,  1988).  Because  the  1996  science  long-term  trend  scale  was  tied  to  the  1986  long-term  trend 
scale  through  the  1990,  1992,  and  1994  data,  the  distribution  of  proficiency  scores  derived  from  the  long- 
term trend  samples  can  be  described  in  terms  of  scale  anchors.  The  five  levels  of  science  proficiency  are 

150  = Knows  everyday  science  facts; 

200  = Understands  simple  scientific  principles; 

250  = Applies  basic  scientific  information; 

300  = Analyzes  scientific  procedures  and  data;  and 

350  = Integrates  specialized  scientific  information. 
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16.6  PARTITIONING  OF  THE  ESTIMATION  ERROR  VARIANCE 


For  each  age,  the  error  variance  of  the  final  reporting  scale  mean  was  partitioned  into  two  parts 
as  described  in  Chapter  11.  One  part  of  the  error  variance  is  due  to  the  sampling  of  students  (sampling 
variance)  and  the  other  is  due  to  the  fact  that  proficiency  0 is  a latent  variable  that  is  estimated  rather  than 
observed.  These  estimates  are  given  in  Table  16-9  (for  stability,  the  estimates  of  the  between-imputation 
variance,  B,  in  Equation  11.9  are  calculated  based  on  100  imputations).  More  detailed  information  for 
gender  and  race/ethnicity  subgroups  is  available  in  Appendix  E. 


Table  16-9 

Estimation  Error  Variance  and  Related  Coefficients 
for  the  Science  Long-Term  Trend  Assessment 

Proportion  of  Variance  Due  To  . . . 


Age 

Total  Estimation 
Error  Variance 

Student  Sampling 

Latency  of  0 

9 

1.13 

0.81 

0.19 

13 

1.04 

0.87 

0.13 

17 

1.40 

0.86 

0.14 
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Chapter  17 


DATA  ANALYSIS  FOR  THE  LONG-TERM  TREND 
WRITING  ASSESSMENT^ 

Eiji  Muraki  and  Bruce  A.  Kaplan 
Educational  Testing  Service 


17.1  INTRODUCTION 

This  chapter  describes  analyses  of  the  writing  prompts^  and  background  items  in  the  1996 
long-term  trend  assessment  of  writing.  These  analyses  led  to  the  results  reported  in  the  NAEP  1996 
Trends  in  Academic  Progress:  Achievement  ofU.S.  Students  in  Science,  1969  to  1996;  Mathematics, 
1973  to  1996;  Reading,  1971  to  1996;  and  Writing,  1984  to  1996  (Campbell,  Voelkl,  & Donahue,  1997). 
Emphasis  is  given  to  the  psychometric  methods  used  to  develop  the  composite  scores  that  formed  the 
basis  of  those  reports. 

The  1996  samples  used  for  the  analysis  of  writing  achievement  are  presented  in  Table  17-1. 
These  samples  comprise  students  selected  both  on  the  basis  of  age  and  grade  in  school.  For  the  writing 
long-term  trend  analysis,  unlike  other  subject-area  long-term  trend  analyses,  only  those  students  selected 
on  the  basis  of  grade  were  included  in  the  reporting  sample,  which  is  referred  to  as  a “grade-only” 
sample.  The  analyses  were  based  on  the  full  age/grade  sample. 


Table  17-1 

NAEP  1996  Long-Term  Trend  Writing  Student  Samples 


Sample 

Booklet 

IDs 

Mode 

Cohort  Assessed 

Time  of 
Testing 

Age 

DeHnition 

Modal 

Grade 

Number 

Assessed 

9 [RW-LTTrend] 

51-56 

Print 

Age  9/Grade  4 

Winter 

CY 

4 

5,019 

13  [RW-LTTrend] 

51-56 

Print 

Age  13/Grade  8 

Fall 

CY 

8 

5,493 

17  [RW-LTTrend] 

51-56 

Print 

Age  17/Grade  11 

Spring 

NotCY 

11 

4,669 

LEGEND 


RW  Reading  and  writing 

LTTrend  Long-term  trend  assessment 
Print  Printed  administration 

CY  Calendar  year:  birthdates  in  1986,  1982,  and  1978,  for  ages  9,  13,  and  17 

Not  CY  Age  17  only:  birthdates  between  October  1 and  September  30  of  the  appropriate  years 


' Eiji  Muraki  was  the  primary  person  responsible  for  the  planning,  specification,  and  coordination  of  the  writing  long-term  trend 
analyses.  Computer  activities  for  all  long-term  trend  writing  scaling  and  data  analyses  were  directed  and  performed  by  Bruce 
Kaplan. 

^ The  terms  “item”  and  “prompt”  are  used  interchangeably  in  this  chapter. 
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The  objectives  of  the  writing  analyses  were  to  prepare  scale  values  and  perform  all  analyses 
necessary  to  produce  a long-term  trend  report  in  writing.  The  writing  long-term  trend  results  include  the 
years  1984,  1988,  1990,  1992,  1994,  and  1996.  Table  17-2  describes  the  samples  of  these  years. 


Table  17-2 

NAEP  Writing  Samples  Contributing  to  1996  Long-Term  Trend  Results,  1984-1996 


Cohort 

Year 

. Sample 

Subjects 

Time  of  Testing 

Mode 

Age  Definition 

Modal  Grade 

Age  9/ 

1984 

Main 

RW 

Winter,  Spring 

Print 

CY 

4 

Grade  4 

1988 

LTTrend^ 

RW 

Winter 

Print 

CY 

4 

1990 

LTTrend^ 

RW 

Winter 

Print 

CY 

4 

1992 

LTTrend^ 

RW 

Winter 

Print 

CY 

4 

1994 

LTTrend^ 

RW 

Winter 

Print 

CY 

4 

1996 

LTTrend^ 

RW 

Winter 

Print 

CY 

4 

Age  13/ 

1984 

Main 

RW 

Winter,  Spring 

Print 

CY 

8 

Grade  8 

1988 

LTTrend^ 

RW 

Fall 

Print 

CY 

8 

1990 

LTTrend^ 

RW 

Fall 

Print 

CY 

8 

1992 

LTTrend^ 

RW 

Fall 

Print 

CY 

8 

1994 

LTTrend^ 

RW 

Fall 

Print 

CY 

8 

1996 

LTTrend^ 

RW 

Fall 

Print 

CY 

8 

Age  17/ 

1984 

Main 

RW 

Winter,  Spring 

Print 

Not  CY 

11 

Grade  11 

1988 

LTTrend^ 

RW 

Spring 

Print 

Not  CY 

11 

1990 

LTTrend^ 

RW 

Spring 

Print 

Not  CY 

11 

1992 

LTTrend^ 

RW 

Spring 

Print 

Not  CY 

11 

1994 

LTTrend^ 

RW 

Spring 

Print 

Not  CY 

11 

1996 

LTTrend^ 

RW 

Spring 

Print 

Not  CY 

11 

^Within  an  age  class  group,  these  samples  received  common  booklets  and  constituted  a trend  line. 


LEGEND 


RW 

Main 

LTTrend 

Print 

CY 

Not  CY 


Reading  and  writing 
Main  assessment 
Long-term  trend  assessment 
Printed  administration 

Calendar  year:  birthdates  in  1986  and  1982,  respectively,  for  ages  9 and  13 
Age  17  only:  birthdates  between  October  1,  1978,  and  September  30,  1979 


As  in  the  1992  and  1994  writing  assessments,  the  IRT  scaling  for  the  1996  assessment  used  a 
model  for  polytomously  scored  items.  The  1992  assessment  established  the  first  such  scale.  The  1994 
assessment  represented  the  first  time  that  the  long-term  trend  writing  scale  had  to  be  linked  to  a 
previously  established  scale.  The  1996  assessment  was  directly  linked  to  the  1994  assessment.  Sampling 
weights,  as  described  in  Chapter  17,  were  used  for  all  analyses  of  the  writing  prornpts. 


17.2  LONG-TERM  TREND  WRITING  DATA  ANALYSIS 

When  IRT  scaling  was  used  for  the  long-term  trend  writing  assessment  for  the  first  time  in  1992, 
a new  scale  was  developed,  and  data  from  the  1984,  1988,  1990,  and  1992  long-term  trend  samples  were 
reanalyzed  to  establish  the  scale.  The  1994  writing  long-term  trend  points  were  linked  to  that  scale,  and 
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the  1996  writing  long-term  trend  points,  based  on  data  from  the  1996  reading/writing  long-term  trend 
samples  (RW-LTTrend  in  Table  17-2),  were  linked  to  those  1994  writing  trend  points.  Each  subsequent 
assessment  matched  the  1984  assessment  in  terms  of  the  time  of  administration  and  age  definitions.  The 
booklets  used  in  this  assessment  contained  blocks  of  reading  and  writing  items,  as  well  as  background 
items.  Identical  booklets  were  used  in  1984,  1988,  1990,  1992,  1994,  and  1996. 

The  items  on  which  the  long-term  trends  in  writing  achievement  are  based  are  shown  in  Table 
17-3.  The  table  shows  the  block  that  contained  the  item  in  1984  and  long-term  trend  booklets  containing 
the  item  in  1988,  1990,  1992,  1994,  and  1996.  Twelve  writing  prompts  were  used  to  measure  long-term 
trends,  with  six  prompts  presented  at  each  grade  level.  To  allow  comparisons  in  writing  ability  across 
grades,  three  of  the  six  prompts  presented  to  fourth-grade  students  were  also  presented  to  eighth-grade 
students;  three  of  the  eighth-grade  prompts  were  also  presented  to  eleventh-grade  students;  and  one  of  the 
common  prompts  (Appleby  House)  was  presented  at  all  three  grade  levels. 


Table  17-3 

Assignment  of  1984-96  Writing  Long-Term  Trend  Items  in 
1984,  1988,  1990,  1992,  1994,  and  1996 


Writing  Task 

1984  BIB-Spiral  Blocks 
Used  for  Long-Term 
Trend  Analyses 
Age  9/  Age  13/  Age  17/ 

Grade  4 Grade  8 Grade  11 

1988-96 

Long-Term 

Trend  Booklet  Numbers 
Age  9/  Age  13/  Age  17/ 

Grade  4 Grade  8 Grade  11 

N0003  Recreation  Opportunity 

C 

C 

52,54 

52,54 

N0004  Food  on  Frontier 

D 

D 

51,54 

51,54 

N0(X)5  Dissecting  Frogs 

E 

53,55 

N0006  XYZ  Company 

E 

E 

52,  54 

53,55 

N0009  Radio  Station 

G 

G 

54,55 

55,56 

NOOIO  Appleby  House 

G 

G 

G 

54,55 

55,56 

55,56 

N0076  Flashlight 

56 

NO  1 47  Plants 

c 

51,53 

N0I48  Spaceship 

E 

52,54 

NO  180  Space  Program 

E 

53,55 

N0190  Job  Application 

E 

53,55 

N0210  Bike  Lane 

G 

55,56 

^Block  V was  not  placed  in  a booklet  with  any  other  writing  block  in  1984  (all  other  blocks  appeared  with  every  other  block  at  the 
same  grade  level  in  1984),  and  hence  could  not  be  used  in  scaling. 


17.2.1  Primary  Trait  Scoring  of  the  Writing  Tasks  and  Measures  of  Scorer  Reliability 

All  writing  exercises  from  the  1996  assessment  were  scored  for  task  accomplishment  (primary 
trait).  For  the  purposes  of  analysis,  the  student  responses  were  coded  as  0 (not  rated),  1 (unsatisfactory), 

2 (minimal),  3 (adequate),  and  4 (elaborated).  Not-reached  and  omitted  items  were  excluded  from  the 
scaling.  The  writing  long-term  trend  blocks  contained  either  one  or  two  items.  If  an  item  was  left  blank  in 
a one-item  block,  it  was  scored  as  an  omission.  Items  considered  not-reached  occurred  only  in  writing 
blocks  that  had  two  cognitive  items  where  the  first  item  was  answered  and  the  second  was  not. 
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A 25  percent  random  subsample  of  all  1996  papers  was  rescored  by  a second  rater  to  provide  an 
estimate  of  interrater  reliability.  Although  the  measures  of  scorer  agreement  in  NAEP  have  been 
consistently  high,  the  possibility  existed  that  there  might  be  variation  between  the  ratings  provided  by  the 
group  of  scorers  assembled  in  1996  and  the  scorers  assembled  in  previous  years.  Such  a variation  would 
be  a confounding  effect  in  trend  measurement.  The  most  direct  way  of  controlling  the  effect  of 
across-year  variation  in  scoring  would  be  to  eliminate  it  entirely  by  rescoring  all  of  the  data  from  the 
previous  four  assessments,  using  the  same  set  of  scorers  who  scored  the  1996  data.  Unfortunately, 
resources  did  not  allow  for  the  rescoring  of  the  full  set  of  writing  papers,  but  did  allow  for  a rescoring  of 
over  1 1,000  of  the  papers  from  1988  (the  numbers  by  prompt  and  grade  are  displayed  in  Table  17-4).  The 
rescored  papers  for  a given  item  constituted  approximately  25  percent  of  all  1988  papers  and  consisted  of 
all  grade-eligible  respondents  to  selected  1988  booklets  containing  that  item.  The  procedure  of  rescoring 
data  from  other  writing  assessments  was  also  used  in  the  previous  years’  assessments  in  the  current  long- 
term trend  analysis  (1984,  1988,  1990,  1992,  and  1994). 

Because  of  rigorous  training  of  scorers,  it  was  expected  that  the  between-year  variability  in 
scoring  would  be  low  enough  to  permit  the  use  of  the  full  set  of  the  1988  data.  Table  17-4  shows  scorer 
reliability,  as  measured  by  the  intraclass  correlation,  for  each  prompt  in  the  1988,  1990,  1992,  1994,  and 
1996  data.  The  percentage  of  exact  agreement  between  first  and  second  raters  is  also  given.  In  addition, 
the  table  shows  the  intraclass  correlation  and  percentage  of  exact  score  agreement  comparing  the  scores 
of  samples  of  the  1990,  1992,  1994,  and  1996  raters  with  those  of  the  1988  raters  on  a sample  of  the  1988 
papers.  The  reliabilities  and  percentages  of  exact  agreement  (between  first  and  second  raters)  were 
generally  high  for  1996  data,  as  they  were  for  prior  assessments  (1984,  1988,  1990,  1992,  and  1994). 


17.3  ITEM  ANALYSIS  FOR  THE  WRITING  LONG-TERM  TREND  ASSESSMENT 

A standard  item  analysis  for  polytomous  items,  as  described  in  Chapter  9,  was  conducted  on  the 
writing  long-term  trend  item  data.  Table  17-5  displays  the  item  analysis  statistics  for  each  grade — the 
number  of  examinees  responding  to  each  prompt,  the  percentage  of  examinees  receiving  each  of  the 
assigned  scores,  and  the  mean  score  of  the  prompt.  R-polyserials  and  alpha  reliabilities  were  not 
calculated  since  there  is  only  one  or  two  items  per  block. 

The  results  of  the  item  analysis  were  examined  to  verify  that  statistics  for  each  item  were  in 
expected  ranges.  No  difficulties  were  found  in  this  process.  Comparisons  of  item  statistics  with  those  of 
previous  years  were  also  made,  and  it  was  found  that  the  items  had  similar  statistics  for  all  five  years  of 
the  long-term  trend  in  writing. 
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Table  17-5 

Descriptive  Statistics  for  Writing  Prompts,  1996  Writing  Long-Term  Trend  Samples 

Weighted  Percentages  of  Grade-Eligible  Examinees  in  Each  Score  Category 
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17.4  IRT  SCALING  FOR  THE  WRITING  LONG-TERM  TREND  ASSESSMENT 


This  section  describes  the  scaling  of  the  primary  trait  data  from  the  1996  writing  long-term  trend 
assessment.  A listing  of  the  prompts  used  in  scaling  at  the  three  grade  levels  is  presented  in  Table  17-6. 
Five  prompts  were  used  at  the  fourth-grade  level  and  six  prompts  were  used  at  each  of  the  eighth-  and 
eleventh-grade  levels.  Either  three  or  four  scoring  categories  were  used  in  the  scaling  for  each  of  the 
prompts.  Three  prompts  (N0006,  N0009,  and  NO  147)  at  the  fourth-grade  level  and  two  (N0006  and 
N0009)  at  the  eighth-grade  level  were  scaled  with  three  categories  because  the  frequencies  of  responses 
in  the  fourth  category  were  zero  or  near  zero.  All  other  prompts  were  scaled  with  four  categories.  After 
examining  the  pattern  of  omitted,  not-reached,  off-task,  and  illegible  responses  relative  to  responses  to 
other  prompts,  it  was  decided  to  treat  these  responses  as  missing,  because  there  appeared  to  be  no 
obvious  relationship  between  writing  performance  and  nonresponse  (for  whatever  reason)  to  other 
prompts.  Treating  such  prompts  as  missing  meant  that  they  would  not  enter  into  the  scaling  process.  One 
of  the  prompts  administered  in  the  assessment,  “Flashlight,”  was  not  administered  with  any  other  prompt, 
and  therefore  could  not  be  put  on  the  same  scale  as  the  other  prompts.  Hence,  this  prompt  was  excluded 
from  scaling  and  from  the  number  of  prompts  cited  above. 

The  resulting  sample  sizes  for  scaling  the  long-term  trend  items  are  provided  in  Table  17-6.  The 
1984  long-term  trend  point  was  based  on  a rescoring  of  a sample  of  the  1984  data  by  the  1988  raters.  For 
details,  see  Johnson  (1990). 

When  the  long-term  trend  items  were  administered  for  the  first  time  in  1984,  they  were  used  as 
part  of  the  1984  BIB  design.  By  applying  the  additional  information  that  was  obtained  from  the  1984 
design,  the  long-term  trend  items  for  each  grade  were  calibrated  together.  This  task  was  not 
straightforward.  Due  to  rater  drift,  the  1984  original  data  for  long-term  trend  items  had  to  be  rescored  in 
1988  and  not  all  the  booklets  were  included  in  the  rescoring.  Although  enough  information  was  captured 
to  calibrate  all  the  fourth-grade  items  together,  this  was  not  the  case  for  the  eighth-  and  eleventh-grade 
items.  There  were  two  non-overlapping  groups  of  items  at  these  two  grade  levels.  Items  within  the  groups 
were  administered  to  common  samples  of  examinees,  but  there  were  no  common  samples  for  items 
between  these  two  groups.  This  prohibited  a direct  linking  via  scaling  all  items  simultaneously.  As  a 
result,  an  additional  step  was  required  for  these  two  grade  levels. 

Age  9/Grade  4 Level.  For  this  level,  the  generalized  partial  credit  model  was  used  to  calibrate, 
simultaneously,  the  prompts  in  the  1994  and  1996  data.  This  required  a single  run  on  the  NAEP 
BILOG/P ARSCALE  computer  program  specifying  two  subpopulations,  one  for  each  year. 

Age  13/Grade  8 and  Age  17/Grade  11  Levels.  For  each  of  these  two  levels,  two  separate 
PARSCALE  computer  runs  were  conducted,  one  on  each  of  the  two  separate  groups  of  prompts 
discussed  above.  Like  the  fourth-grade  level,  the  1994  and  1996  data  were  simultaneously  scaled  using 
two  subpopulations. 
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• Table  17-6 

Age/Grade  Sample  Sizes  for  Scaling  Items  Used  in  Long-Term  Trends  in  Writing  Performance^ 

Age  9/Grade  4 Age  13/Grade  8 Age  17/Grade  11 

1984  1984  1984 

NAEP  Item  Rescore  1988  1990  1992  1994  1996  Rescore  1988  1990  1992  1994  1996  Rescore  1988  1990  1992  1994  1996 


17.4.1  Item  Parameter  Estimation  Using  the  Generalized  Partial  Credit  Model 


The  first  step  in  the  scaling  process  was  the  estimation  of  item  parameters  for  the  long-term  trend 
items.  This  item  calibration  was  performed  using  the  BILOG/PARSCALE  program  described  in  Chapter 
11.  Items  were  calibrated  separately  for  each  of  the  three  age/grade  groups.  Item  parameters  were 
estimated  using  combined  data  from  the  assessment  years  1994  and  1996,  treating  each  assessment  as  a 
sample  from  a separate  subpopulation.  In  age  13/grade  8 and  age  17/grade  1 1,  there  were  two  sets  of 
nonoverlapping  items.  Each  set  was  scaled  separately  (see  Table  17-7).  Therefore,  there  were  five  sets  of 
calibrations— one  for  age  9/grade  4,  and  two  each  for  age  13/grade  8 and  age  17/grade  11.  Student 
weights  were  used  for  the  calibration.  To  ensure  that  each  assessment  year  had  a similar  influence  on  the 
calibration,  student  weights  for  1994  and  1996  examinees  were  multiplied  by  a constant  in  order  to  adjust 
them  to  have  the  same  sum,  arbitrarily  placed  at  1,000.  Approximately  1,200-1,800  examinees  were 
present  in  each  assessment  year  for  each  item. 


Table  17-7 

Nonoyerlapping  Sets  of  Items  for  the  Writing  Long-Term  Trend  Assessment 


Age/Grade 

Set 

Booklets 

Items 

9/4 

A 

51-55 

N0006,  N0009,  NOOlO,  N0147,  N0148 

13/8 

A 

51,52,  54 

N0003,  N0004 

B 

53,  55,  56 

N0005,  N0006,  N0009,  NOOlO 

17/11 

A 

51,52,  54 

N0003,  N0004 

B 

53,  55,  56 

NOOlO,  NO  180,  NO  190,  N0210 

The  final  1994  item  parameter  estimates  were  used  as  starting  values.  The  proficiency 
distribution  of  each  assessment  year  was  constrained  to  be  normally  distributed,  although  the  means  and 
variances  differed  across  assessment  years.  Calibration  was  concluded  when  changes  in  item  parameters 
became  negligibly  small  (i.e.,  less  than  .005).  The  item  parameter  estimates  appear  in  Appendix  D. 


17.5  GENERATION  OF  PLAUSIBLE  VALUES 

The  generation  of  plausible  values  was  conducted  independently  for  each  set  of  items  within  an 
age/grade  level  for  each  of  the  assessment  years.  The  item  parameters  from  BILOG/PARSCALE,  final  . 
student  weights,  item  responses,  and  selected  background  variables  were  used  with  the  computer  program 
BGROUP  (described  in  Chapter  1 1)  to  generate  the  values  for  each  age/grade  level.  The  background 
variables  included  student  demographic  characteristics  (e.g.,  race/ethnicity  of  the  student,  highest  level  of 
education  attained  by  parents),  students’  perceptions  about  writing,  and  student  behavior  both  in  and  out 
of  school  (e.g.,  amount  of  television  watched  daily,  amount  of  homework  done  each  day).  Appendix  C 
gives  the  codings  for  the  conditioning  variables  for  the  three  age/grade  groups.  Table  17-8  contains  a list 
of  the  number  of  background  variables  included  in  conditioning,  as  well  as  the  proportion  of  variance 
accounted  for  by  the  conditioning  model  for  each  age/grade. 
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Table  17-8 

Proportion  of  Proficiency  Variance  Accounted  for 
by  the  Conditioning  Model  for  the  Writing  Long-Term  Trend  Assessment 


Age/Grade 

Number  of 

ConditioningConstants^ 

Proportion 
of  Variance 

9/4 

48 

0.43 

13/8 

48 

0.55 

17/11 

48 

0.38 

^ Excluding  the  constant  term. 


17.6  THE  FINAL  WRITING  LONG-TERM  TREND  SCALE 


The  linear  indetemiinacy  of  the  long-term  trend  scale  was  resolved  by  linking  the  1996  trend 
scales  to  previous  trend  scales.  For  sets  within  each  age/grade,  the  item  parameters  from  the  joint 
calibration  based  on  data  from  1994  and  1996  were  used  with  the  1994  data  to  find  plausible  values  for 
the  1996  data.  The  mean  and  standard  deviation  for  all  of  the  plausible  values  were  calculated  and 
matched  to  the  mean  and  standard  deviation  for  all  of  the  plausible  values,  based  on  the  original  analysis 
of  the  1994  data,  as  given  in  earlier  reports.  The  transformations  that  resulted  from  this  matching  of  the 
first  two  moments  for  the  1994  data  are 


Age  9/Grade  4: 

Age  13/Grade  8 A: 
Age  13/Grade  8B: 
Age  17/Grade  HA: 
Age  17/Grade  IIB: 


0ta.^et  = 39.15  •0ealibrated  + 202.21, 
0target  = 37.61  •0calibrated  + 258.92, 
0target=  37.04  •0calibrated  + 260.29, 

= 36.33  • 0caiibrated  + 280.48,  and 

0target=  34.75  •0calibrated  + 279.66, 


where  0iarget  denotes  values  on  the  final  transformed  scale  and  0caiibrated  denotes  values  on  the  calibration 
scale. 


As  described  above,  the  IRT  parameters  are  estimated,  plausible  values  computed,  and  final  scale 
transformations  established  using  the  age/grade  samples.  Reporting  was  done,  however,  using  grade-only 
samples.  Overall  summary  statistics  for  the  long-term  trend  grade  samples  are  given  in  Table  17-9. 
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Table  17-9 

Means  and  Standard  Deviations  on  the  Writing  Long-Term  Trend  Scale 


Grade 

Assessment  Year 

All  Five  Plausible  Values 
Mean  S.D. 

4 

1984 

203.8 

36.5 

1988 

205.7 

42.0 

1990 

201.7 

41.7 

1992 

207.1 

38.3 

1994 

204.8 

38.3 

1996 

207.4 

38.1 

8 

1984 

266.7 

29.5 

1988 

263.7 

32.4 

1990 

256.6 

37.5 

1992 

274.4 

36.3 

1994 

265.3 

35.9 

1996 

263.6 

36.4 

11 

1984 

289.7 

31.8 

1988 

291.3 

27.9 

1990 

287.1 

36.5 

1992 

287.3 

32.0 

1994 

284.6 

34.4 

1996 

283.0 

34.6 

17.7  PARTITIONING  OF  ESTIMATION  ERROR  VARIANCE 

The  variance  of  proficiency  means  for  each  age/grade  level  was  partitioned  into  the  part  due  to 
the  sampling  of  students  and  the  part  due  to  the  latency  of  proficiency,  0,  as  described  in  Chapter  17. 
These  estimates  are  given  in  Table  17-10  (for  stability,  the  estimates  of  the  between-imputation  variance, 
B,  in  Equation  1 1.9  are  computed  based  on  100  imputations).  More  detailed  information  is  available  for 
gender  and  race/ethnicity  subgroups  in  Appendix  E. 


Table  17-10 

Estimation  Error  Variance  and  Related  Coefficients 
for  the  Writing  Long-Term  Trend  Assessment 


Age/Grade 

Total  Estimation  Error  Variance 

Proportion  of  Variance  due  to  . . . 
Student  Sampling  Latency  of  6 

9/4 

0.94 

0.89 

0.11 

13/8 

0.95 

0.89 

0.11 

17/11 

1.24 

0.94 

0.06 
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Chapter  18 


CONVENTIONS  USED  IN  HYPOTHESIS  TESTING 
AND  REPORTING  NAEP  RESULTS^ 

Spencer  S.  Swinton,  David  S.  Freund,  and  Nancy  L.  Allen 
Educational  Testing  Service 


18.1  OVERVIEW 

Results  for  the  1996  NAEP  Assessments  were  disserninated  in  several  different  reports:  the 
NAEP  1996  Mathematics  Report  Card  for  the  Nation  and  the  States,  the  NAEP  1996  Science  Report 
Card  for  the  Nation  and  the  States,  NAEP  1996  Trends  in  Academic  Progress,  Cross-State  Data 
Compendium  from  the  NAEP  1996  Mathematics  Assessment,  Cross-State  Compendium  from  the  NAEP 
1996  Science  Assessment,  and,  distributed  only  in  electronic  form,  six  sections  of  summary  data  tables 
for  each  report.  These  reports  are  published  on  the  NCES/NAEP  website  http://nces.ed.gov/naep.  Several 
other  reports  based  on  1996  NAEP  data  will  be  forthcoming. 

The  NAEP  1996  Mathematics  Report  Card  for  the  Nation  and  the  States  and  the  NAEP  1996 
Science  Report  Card  for  the  Nation  and  the  States  highlight  key  assessment  results  for  the  nation  and 
summarize  results  across  the  jurisdictions  participating  in  the  assessments.  These  reports  contain 
composite  scale  score  results  (scale  score  means,  etc.)  for  the  nation,  for  each  of  the  four  regions  of  the 
country,  and  for  public-school  students  within  each  jurisdiction  participating  in  the  State  Assessment  , 
both  overall  and  by  primary  reporting  variables.  The  seven  key  reporting  variables  (referred  to  here  as 
primary  reporting  variables)  are  gender,  race/ethnicity,  level  of  parents’  education.  Title  I participation, 
eligibility  for  free  or  reduced  cost  school  lunch,  type  of  location,  and  type  of  school  (public.  Catholic 
schools,  other  religious  schools,  and  other  private  schools).  For  public-school  students,  scale  score  means 
were  reported  for  a variety  of  other  subpopulations  defined  by  responses  to  items  from  the  student, 
teacher,  and  school  questionnaires  and  by  school  and  location  demographic  variables  provided  by 
Westat,  Inc.^  Upcoming  reports  will  include  estimates  of  scale  score  means  and  selected  percentiles  for 
specific  subgroups  of  students  of  interest  in  each  report. 

The  report  NAEP  1996  Trends  in  Academic  Progress  provides  a look  at  NAEP  results  for 
Science,  Mathematics,  Reading,  and  Writing  since  the  first  NAEP  assessments  of  those  subjects  in  1969- 
70.  This  report  includes  scale  score  results  for  the  nation  overall  and  by  gender,  race/ethnicity,  gender 
and  race/ethnicity,  region,  level  of  parents’  education  and  type  of  school  (public  and  nonpublic).  It  also 
provides  percentages  of  students  in  categories  defined  by  subject  specific  background  variables  (such  as 
students  who  reported  having  experimented  with  living  plants),  along  with  their  average  scale  scores.  The 
report  contains  trends  in  average  scale  scores  by  quartile  and  percentages  of  students  performing  at  or 
above  selected' performance  levels.  An  additional  report  gives  data  for  the  mechanics  of  writing  long- 
term trend. 

' Spencer  S.  Swinton  played  a role  in  making  decisions  about  hypothesis  testing  methods  and  procedures  and  worked  with  David 
S.  Freund  who  implemented  many  of  the  methods  and  procedures  in  computer  programs.  Nancy  L.  Allen  contributed  to  the 
current  version  of  this  chapter. 

^ Further  technical  documentation  for  the  State  Assessments  appears  in  the  Technical  Report  of  the  NAEP  1996  State  Assessment 
Program  in  Mathematics  and  the  Technical  Report  of  the  NAEP  1996  State  Assessment  Program  in  Science. 

^ Some  of  these  variables  were  used  by  Westat,  Inc.,  in  developing  the  sampling  frame  for  the  assessment  and  in  drawing  the 
sample  of  participating  schools. 
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The  third  type  of  report  consists  of  a number  of  data  compendia.  Two  of  these  are  entitled  the 
Cross-State  Data  Compendium  from  the  NAEP  1996  Mathematics  Assessment  and  the  Cross-State  Data 
Compendium  from  the  NAEP  1996  Science  Assessment.  Like  the  Report  Cards,  the  Compendia  report 
results  for  the  nation  and  for  all  of  the  jurisdictions  participating  in  the  State  Assessment.  The 
Compendia  contain  most  of  the  tables  included  in  the  Report  Cards  plus  tables  that  provide  composite 
scale  results  for  a large  number  of  secondary  reporting  variables  (e.g.,  amount  of  homework,  teacher 
preparation). 

The  fourth  type  of  summary  report  is  an  electronically-delivered  collection  of  summary  data 
tables  that  contain  detailed  breakdowns  of  the  science  scale  score  data  for  each  sarnple  according  to  the 
responses  to  the  student,  teacher,  and  school  questionnaires  for  the  public-school,  nonpublic-school,  and 
combined  populations  as  a whole  and  for  important  subgroups  of  the  public-school  population,  as  defined 
by  the  primary  reporting  variables.  There  are  six  sections  in  each  collection  of  summary  data  tables! 

The  Distribution  Data  Section  provides  selected  composite-scale  and  science  subscale 
percentiles  for  the  public-school,  nonpublic-school,  and  total  populations  and  for  the 
major  demographic  subgroups  of  the  national  school  population. 

The  Student  Questionnaire  Section  breaks  down  the  composite  scale  score  data  according 
to  the  students  responses  to  questions  in  the  three  student  questionnaires  (common  core, 
subject-specific  background,  and  motivational  section)  included  in  the  assessment 
booklets. 

The  Teacher  Questionnaire  Section  breaks  down  the  composite  scale  score  data 
according  to  the  teachers  responses  to  questions  in  teacher  questionnaires,  where  they 
are  available. 

The  School  Questionnaire  Section  breaks  down  the  composite  scale  score  data  according 
to  the  principals’  (or  other  administrators’)  responses  to  questions  in  the  school 
characteristics  and  policies  questionnaire. 

The  Scale  Section  breaks  down  the  scale  score  data  for  the  mathematics  content  strands 
or  the  fields  of  science  according  to  selected  items  (such  as  the  amount  of  science 
homework  done  per  day)  from  the  questionnaires. 

The  Item  Section  provides  the  response  data  (percent  of  students  choosing  each  option) 
for  each  cognitive  item  in  the  assessment. 

The  production  of  these  reports  required  many  decisions  about  a variety  of  data  analysis  and 
statisdcal  issues.  For  example,  certain  categories  of  the  reporting  variables  contained  limited  numbers  of 
examinees.  A decision  was  needed  as  to  what  constituted  a sufficient  sample  size  to  permit  the  reliable 
reporting  of  subgroup  results,  and  which,  if  any,  estimates  were  sufficiently  unreliable  to  need  to  be 
“flagged”  as  a caution  to  readers.  As  a second  example,  the  performance  for  subgroups  of  students  were 
compared.  A number  of  inferential  rules,  based  on  logical  and  statistical  considerations,  had  to  be 
developed  to  ensure  that  conclusions  are  adequately  supported  by  the  data  from  the  assessment.  Practical 
comparison  procedures  were  required  to  control  for  Type  I errors  without  paying  too  large  a penalty  with 
respect  to  the  statistical  power  for  detecting  real  and  substantively  interesting  differences.  For  most  tests, 
the  number  of  related  tests  was  not  so  large  that  the  Bonferroni  test  (Hochberg,  1988)  exacted  too  large  a 
penalty  in  power  in  exchange  for  protection  from  Type  I error.  For  sets  of  comparisons  with  very  large 
numbers  of  related  tests,  such  as  comparing  a state  to  all  other  states,  a new  multiple  comparison 
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criterion,  False  Discovery  Rate  or  FDR  (Benjamini  & Hochberg,  1994),  was  implemented.  FDR  controls 
the  rate  of  false  rejections  (e.g.,  five  false  rejections  per  100  rejections),  rather  than  controlling  the 
probability  of  one  such  error  (Familywise  Error  Rate,  or  FWE),  as  the  Bonferroni  procedure  does. 

The  purpose  of  this  chapter  is  to  document  the  major  conventions  and  statistical  procedures  used 
in  generating  the  Report  Cards,  NAEP  1996  Trends,  the  Data  Compendia,  and  the  summary  data  tables. 
Additional  details  about  procedures  relevant  to  the  Report  Card  and  Cross-State  Data  Compendia  can  be 
found  in  the  text  and  technical  appendices  of  those  reports. 


18.2  MINIMUM  SCHOOL  AND  STUDENT  SAMPLE  SIZES  FOR  REPORTING 
SUBGROUP  RESULTS 

In  all  of  the  reports,  estimates  of  quantities  such  as  composite  and  scale  score  means  and 
percentages  of  students  indicating  particular  levels  of  background  variables  (as  measured  in  the  student, 
teacher,  and  school  questionnaires)  are  reported  for  the  population  of  students  in  each  grade.  These 
estimates  are  also  reported  for  certain  key  subgroups  of  interest  as  defined  by  primary  NAEP  reporting 
variables.  Where  possible,  NAEP  reports  results  for  gender,  for  five  racial/ethnic  subgroups  (White, 

Black,  Hispanic,  Asian  American/Pacific  Islander,  and  American  Indian/ Alaskan  Native),  three  types  of 
locations  (central  cities,  urban  fringes/large  towns,  rural/small  town  areas),  four  levels  of  parents 
education  (did  not  finish  high  school,  high  school  graduate,  some  college,  college  graduate).  Title  1 
participation,  eligibility  for  the  free  or  reduced-cost  school  lunch  component  of  the  National  School 
Lunch  Program,  and  type  of  school.  However,  for  some  regions  of  the  country  and  sometimes  for  the 
nation  as  a whole,  school  and/or  student  sample  sizes  were  too  small  for  one  or  more  of  the  categories  of 
these  variables  to  permit  accurate  reporting. 

A consideration  in  deciding  whether  to  report  an  estimated  quantity  is  whether  the  sampling  error 
is  too  large  to  permit  effective  use  of  the  estimates.  A second,  and  equally  important,  consideration  is 
whether  the  standard  error  estimate  that  accompanies  a statistic  is  itself  sufficiently  accurate  to  inform 
potential  readers  about  the  reliability  of  the  statistic.  The  precision  of  a sample  estimate  (be  it  sample 
mean  or  standard  error  estimate)  for  a population  subgroup  from  a three-stage  sample  design  (the  one 
used  to  select  samples  for  the  national  assessments)  is  a function  of  the  sample  size  of  the  subgroup  and 
of  the  distribution  of  that  sample  across  first-stage  sampling  units  (i.e.,  PSUs  in  the  case  of  the  national 
assessments).  Hence,  both  of  these  factors  were  used  in  establishing  minimum  sample  sizes  for  reporting. 

Here  a decision  was  reached  to  report  subgroup  results  only  if  the  student  sample  size  exceeded 
61.^  A design  effect  of  two  was  assumed  for  this  decision,  implying  a sample  design-based  variance  twice 
that  of  simple  random  sampling.  This  assumption  is  consistent  with  previous  NAEP  experience  (Johnson 
& Rust,  1992).  In  carrying  out  the  statistical  power  calculations  when  comparing  a subgroup  to  the  total 
group,  it  was  assumed  that  the  total  population  sample  size  is  large  enough  to  contribute  negligibly  to 
standard  errors.  Furthermore,  it  was  required  that  the  students  within  a subgroup  be  adequately 
distributed  across  PSUs  to  allow  for  reasonably  accurate  estimation  of  standard  errors.  In  consultation 
with  Westat,  a decision  was  reached  to  publish  only  those  statistics  that  had  standard  error  estimates 
based  on  five  or  more  degrees  of  freedom.  The  same  minimum  student  and  PSU  sample  size  restrictions 
were  applied  to  proportions  and  to  comparisons  of  percentages  or  proportions  as  well  as  average  scale 
scores  and  comparisons  of  average  scale  scores. 


’ This  number  was  obtained  by  determining  the  sample  size  necessary  to  detect  an  effect  size  of  0.5  with  a probability  of  0.8  or 
greater. 
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18.3  IDENTIFYING  ESTIMATES  OF  STANDARD  ERRORS  WITH  LARGE  MEAN 
SQUARED  ERRORS 

As  noted  above,  standard  errors  of  average  scale  scores,  proportions,  and  percentiles  play  an 
important  role  in  interpreting  subgroup  results  and  in  comparing  the  performances  of  two  or  more 
subgroups.  The  jackknife  standard  errors  reported  by  NAEP  are  statistics  whose  quality  depends  on 
certain  features  of  the  sample  from  which  the  estimate  is  obtained.  In  certain  cases,  the  mean  squared 
error®  associated  with  the  estimated  standard  errors  may  be  quite  large.  This  result  typically  occurred 
when  the  number  of  students  upon  which  the  standard  error  is  based  is  small  or  when  this  group  of 
students  comes  from  a small  number  of  participating  PSUs.  The  minimum  PSU  and  student  sample  sizes 
that  were  imposed  in  most  instances  suppressed  statistics  where  such  problems  existed.  However,  the 
possibility  remained  that  some  statistics  based  on  sample  sizes  that  exceed  the  minimum  requirements 
had  standard  errors  that  were  not  well  estimated.  Therefore,  in  the  reports,  estimated  standard  errors  for 
published  statistics  that  are  themselves  subject  to  large  mean  squared  errors  are  followed  by  the  symbol 


The  magnitude  of  the  mean  squared  error  associated  with  an  estimated  standard  error  for  the 
mean  or  proportion  of  a group  depends  on  the  coefficient  of  variation  (CV)  of  the  estimated  size  of  the 

population  group,  denoted  as  N (Cochran,  1977,  Section  6.3).  The  coefficient  of  variation  is  estimated 
by: 


CV(N)  = 


SE(N) 

N 


where  A is  a point  estimate  of  N and  SE(N)  is  the  jackknife  standard  error  (described  in  Chapter  10  of 
this  report)  of  N. 

Experience  with  previous  NAEP  assessments  suggests  that  when  this  coefficient  exceeds  0.2,  the 
mean  squared  error  of  the  estimated  standard  errors  of  means  and  proportions  based  on  samples  of  this 
size  may  be  quite  large.  (Further  discussion  of  this  issue  can  be  found  in  Johnson  & Rust,  1992.) 
Therefore,  the  standard  errors  of  means  and  proportions  for  all  subgroups  for  which  the  coefficient  of 
variation  of  the  population  size  exceeds  0.2  are  marked  as  described  above.  In  the  Report  Cards,  NAEP 
Trends,  the  Data  Compendia,  and  the  summary  data  tables,  statistical  tests  involving  one  or  more 
quantities  that  have  standard  errors,  confidence  intervals,  or  significance  tests  so  flagged  should  be 
interpreted  with  caution. 


18.4  TREATMENT  OF  MISSING  DATA  FROM  THE  STUDENT,  TEACHER,  AND 
SCHOOL  QUESTIONNAIRES 

As  previously  described,  responses  to  the  student,  teacher,  and  school  questionnaires  played  a 
prominent  role  in  all  reports.  Although  the  return  rate  on  all  three  types  of  questionnaire  was  high,^  there 
were  missing  data  for  each  type  of  questionnaire. 


The  mean  squared  error  of  the  estimated  standard  error  is  defined  as  S - O ]^ , where  S is  the  estimated  standard  error.O  is 
the  “true”  standard  error,  and  ^ is  the  expectation,  or  expected  value  operator. 

7 

Information  about  survey  participation  rates  (both  school  and  student),  as  well  as  proportions  of  students  excluded  by  each 
jurisdiction  from  the  assessment,  is  given  in  Appendix  B.  Sampling  adjustments  intended  to  account  for  school  and  student 
nonresponse  are  described  in  Chapter  7. 
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The  reported  estimated  percentages  of  students  in  the  various  categories  of  background 
variables,  and  the  estimates  of  the  average  scale  score  of  such  groups,  were  based  on  only  those  students 
for  whom  data  on  the  background  variable  were  available.  In  the  terminology  of  Little  and  Rubin  (1987), 
the  analyses  pertaining  to  a particular  background  variable  presented  in  the  reports  are  contingent  on  the 
assumption  that  the  data  are  missing  completely  at  random.* 

The  estimates  of  proportions  and  proficiencies  based  on  “missing-completely-at-random” 
assumptions  are  subject  to  potential  nonresponse  bias  if,  as  may  be  the  case,  the  assumptions  are  not 
correct.  The  amount  of  missing  data  was  small  (usually,  less  than  2%)  for  most  of  the  variables  obtained 
from  the  student,  school,  and  teacher  questionnaires.  For  analyses  based  on  these  variables,  reported 
results  are  subject  to  little,  if  any,  nonresponse  bias.  However,  for  particular  background  items  from  the 
student,  school,  and  teacher  questionnaires,  the  level  of  nonresponse  was  somewhat  higher.  As  a result, 
the  potential  for  nonresponse  bias  in  the  results  of  analyses  based  on  this  latter  set  of  background  items  is 
also  somewhat  greater.  Background  items  for  which  more  than  10  percent  of  the  returned  questionnaires 
were  missing  are  identified  in  the  questionnaire  sections  (as  specified  at  the  beginning  of  this  chapter)  of 
the  summary  data  tables.  Again,  results  for  analyses  involving  these  background  variables  should  be 
interpreted  with  caution. 

To  analyze  the  relationships  among  teachers’  questionnaire  responses  and  their  students’ 
achievement,  each  teacher’s  questionnaire  had  to  be  matched  to  the  students  who  were  taught  by  that 
teacher.  If  a student  could  not  be  matched  to  a teacher,  all  teacher  questionnaire  responses  are  missing  for 
that  student.  The  percentages  of  students  that  were  matched  to  teacher  questionnaires  in  each  sample  for 
which  a teacher  questionnaire  was  administered  are  reported  in  the  subject  area  Chapters  12  and  13. 
Lower  percentages  of  students  with  teacher  questionnaire  data  indicate  that  there  is  less  certainty  about 
results  for  variables  from  the  teacher  questionnaire.  Note  that  these  match  rates  do  not  reflect  the 
additional  missing  data  due  to  item-level  nonresponse.  The  amount  of  additional  item-level  nonresponse 
in  the  returned  teacher  questionnaires  can  be  found  in  the  summary  data  tables. 


18.5  HYPOTHESIS  TESTING  CONVENTIONS 

18.5.1  Comparing  Means  and  Proportions  for  Different  Groups  of  Students 

Many  of  the  group  comparisons  explicitly  commented  on  in  the  reports  involved  mutually 
exclusive  sets  of  students.  Examples  include  comparisons  of  the  average  scale  score  for  male  and  female 
students.  White  and  Hispanic  students,  students  attending  schools  in  central  city  and  urban  fringe/large 
town  locations,  students  who  reported  watching  six  or  more  hours  of  television  each  night  and  students 
who  report  watching  less  than  one  hour  each  night. 

The  text  in  the  reports  indicate  that  means  or  proportions  from  two  groups  were  different  only 
when  the  difference  in  the  point  estimates  for  the  groups  being  compared  was  statistically  significant  at 
an  approximate  simultaneous  oc  level  of  0.05.  An  approximate  procedure  was  used  for  determining 
statistical  significance  NAEP  staff  judged  to  be  statistically  defensible,  as  well  as  being  computationally 
tractable.  Although  all  pairs  of  levels  within  a variable  were  tested  and  reported  in  the  summary  data 
tables,  some  text  within  the  reports  was  developed  for  only  a subset  of  these  comparisons  although  the 
family  size  was  maintained  at  that  of  the  original  tests.  For  example,  text  was  included  in  the  reports  to 
compare  the  majority  ethnic  group  and  each  minority  group,  but  text  for  all  possible  comparisons  of 
groups  may  not  have  been  included. 

* The  mechanism  generating  the  missing  data  is  independent  of  both  the  response  to  the  particular  background  items  and  the 
scale  score. 
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The  procedure  used  to  make  statistical  tests  is  described  in  the  following  paragraphs.  This 
procedure  was  used  in  all  cases  except  when  comparisons  were  made  with  students  assessed  in 
assessment  years  for  which  average  scale  scores  were  extrapolated  as  part  of  the  long-term  trend 
analyses.  In  those  cases,  z-tests  comparing  the  test  statistics  to  the  appropriate  value  from  the  standard 
normal  distribution  was  used. 

Let  Ai  be  the  statistic  in  question  (e.g.,  a mean  for  group  i)  and  let  Sai  be  the  jackknife  standard 

error  of  the  statistic.  The  text  in  the  reports  identified  the  means  or  proportions  for  groups  i andy  as  being 
different  if; 


I A,  - Aj  I 

Vsi,  (A,)  + si  {Aj]  - f 


where  is  the  (1  - oc)  percentile  of  the  t distribution  with  degrees  of  freedom,  df,  as  estimated  below, 
and  c is  the  number  of  related  comparisons  being  tested.  See  the  following  section  (Section  18.5.2)  for  a 
more  specific  description  of  multiple  comparisons.  In  cases  where  group  comparisons  were  treated  as 
individual  units,  the  value  of  c was  taken  as  1,  and  the  test  statistic  was  approximately  equivalent  to  a standard 
two-tailed  t-test  for  the  difference  between  group  means  or  proportions  from  large  independent  samples  with 
the  oc  level  set  at  0.05.  When  c is  not  1,  this  test  is  based  on  the  Bonferroni  procedure  described  in  Hochberg 
(1988).  The  degrees  of  freedom  of  this  t-test  is  defined  by  a Satterthwaite  (Johnson  & Rust,  1992) 
approximation  as  follows; 


N 


where  N is  the  number  of  subgroups  involved,  and  djA,,  is  as  follows; 


J=i 


where  m is  the  number  of  replicates,  tj  is  they*  replicated  estimate  for  the  mean  of  a subgroup,  and  t is  the 
estimate  of  the  subgroup  mean  using  the  overall  weights  and  the  first  plausible  value. 

The  procedures  in  this  section  assume  that  the  data  being  compared  are  from  independent 
samples.  Because  of  the  sampling  design  in  which  PSUs,  schools,  and  students  within  school  are 
randomly  sampled,  the  data  from  mutually  exclusive  sets  of  students  may  not  be  strictly  independent. 
Therefore,  the  significance  tests  employed  are,  in  many  cases,  only  approximate.  As  described  in  Section 
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10.4,  another  procedure,  one  that  does  not  assume  independence,  could  have  been  conducted.  However, 
that  procedure  is  computationally  burdensome.  A comparison  of  the  standard  errors  using  the 
independence  assumption  and  the  correlated  group  assumption  was  made  using  NAEP  data.  The 
estimated  standard  error  of  the  difference  based  on  independence  assumptions  was  approximately  ten 
percent  larger  than  the  more  complicated  estimate  based  on  correlated  groups.  In  almost  every  case,  the 
correlation  of  NAEP  data  across  groups  was  positive.  Because,  in  NAEP,  significance  tests  based  on 
assumptions  of  independent  samples  are  only  somewhat  conservative,  the  approximate  procedure  was 
used  for  most  comparisons. 

The  procedures  described  above  were  used  for  testing  differences  of  both  means  and  nonextreme 
percentages.  The  approximation  for  the  test  for  percentages  works  best  when  sample  sizes  are  large,  and 
the  percentages  being  tested  have  magnitude  relatively  close  to  50  percent.  Statements  about  group 
differences  should  be  interpreted  with  caution  if  at  least  one  of  the  groups  being  compared  is  small  in 
size  and/or  if  “extreme”  percentages  are  being  compared.  Differences  in  percentages  were  treated  as 
involving  “extreme”  percentages  if  for  either  percentage,  P: 

200  P(100^P} 

P < Piim  = , where  the  effective  sample  size  is  N^ff  — — — 3 — > ^nd  SEjf^  is  the  jackknife 

/V^yr/r  “b  2 \^PjkJ 

Standard  error  of  P.  Similarly,  at  the  other  end  of  the  0 - 100  scale,  a percentage  is  deemed  extreme  if  100 
-P  < Piim-  In  either  extreme  case,  the  normal  approximation  to  the  distribution  is  a poor  approximation, 
and  the  value  of  P was  reported,  but  no  standard  error  was  estimated  and  hence  no  significance  tests  were 
conducted. 


18.5.2  Multiple  Comparison  Procedures 

Frequently,  groups  (or  families)  of  comparisons  were  made  and  were  presented  as  a single  set. 
The  appropriate  text,  usually  a set  of  sentences  or  a paragraph,  was  selected  for  inclusion  in  a report 
based  on  the  results  for  the  entire  set  of  comparisons.  For  example,  some  reports  contain  a section  that 
compared  average  scale  scores  for  a predetermined  group,  generally  the  majority  group  (in  the  case  of 
race/ethnicity,  for  example.  White  students)  to  those  obtained  by  other  minority  groups.  The  entire  set  of 
tests  was  presented  in  the  summary  data  tables.  For  families  of  comparisons  like  these,  a Bonferroni 
procedure  (Miller,  1966),  controlling  the  Familywise  Error  Rate  (FWE),  was  used.  This  procedure 
defines  the  value  of  To,  as  in  the  previous  section,  where  c is  the  number  of  contrasts  in  the  set.  In  the 
race/ethnicity  example,  c was  taken  to  be  the  number  of  minority  groups  meeting  minimum  sample  size 
requirements,  and  each  statistical  test  was  consequently  carried  out  at  an  a level  of  0.05/c. 

However,  in  an  attempt  to  gain  greater  power,  two  separate  definitions  of  family  size  were 
employed  for  comparisons  in  two-way  tables.  For  n levels  of  a control  variable  (e.g.,  ethnicity)  and  m 
levels  of  a comparison  variable  (e.g.,  number  of  hours  of  homework),  the  standard  Bonferroni  family  size 
of  n X m X (m-l)/2  was  used.  In  addition,  when  the  m X (m-l)/2  marginal  tests  yielded  a significant 
difference  for  a pair  of  categories  of  the  comparison  variable,  the  n levels  of  the  control  variable 
corresponding  to  that  pair  of  categories  were  tested  with  a family  size  of  n.  Significance  was  reported  if 
either  definition  of  family  size  met  the  criterion. 

Further,  in  the  Report  Card  and  summary  data  tables,  two-way  interactions  were  tested  directly 
for  some  variables.  The  tests  for  an  m x n table  were  t-tests  using  a family  size  n x (n-1)  x m x (m-l)/4.  In 
these  cases,  a modification  due  to  Hochberg  of  the  standard  Bonferroni  procedure  was  employed,  in 
which  probabilities  associated  with  outcomes  are  ordered,  and  a is  divided  by  an  integer  which  increases 
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from  ! to  the  family  size  as  successively  smaller  probabilities  are  tested.  More  formally,  the  Hochberg 
Stagewise  Procedure  (Hochberg,  1988)  is  defined  as  follows: 

Let  q be  the  number  of  significance  tests  made  (the  family  size)  and  let  P,  <P2  <...  <P„  be  the 
ordered  significance  levels  for  the  q tests.  Let  arbe  the  combined  significance  level.  The  Hochberg 
procedure  compares  P,  with  a,  P^.,  with  a/2,  Pj  with  a/(q-j+l),  stopping  comparisons  with  the  first  j 
such  that  Pj  < a/(q—j+l).  All  tests  associated  with  P/,  ...,  Py  are  declared  significant;  all  tests  associated 
with  Py+/,  ...,  P^  are  declared  nonsignificant. 

To  compare  a jurisdiction  in  a State  Assessment  with  the  nation  and  all  other  participating 
jurisdictions,  as  many  as  46  different  comparisons  need  to  be  computed.  This  is  done  in  the  comparisons 
of  overall  scale  score  maps  in  the  State  Assessment  reports  and  in  the  comparisons  of  short-term  trends  in 
mathematics  achievement  in  the  Mathematics  Report  Card.  A potentially  more  powerful  multiple 
comparison  procedure  was  used  to  judge  significance  in  this  case.  The  procedure,  described  by 
Benjamini  and  Hochberg  (1994),  was  the  procedure  chosen.  Unlike  the  Bonferroni  procedure  that 
controls  the  FWE,  the  procedure  described  by  Benjamini  and  Hochberg  (1994)  controls  the  expected 
proportion  of  falsely  rejected  hypotheses  among  all  rejections  (FDR).  For  example,  at  the  0.05  level,  for 
every  100  rejections  of  the  null  hypothesis,  the  procedure  ensures  that  no  more  than  five  will  be  expected 
to  be  false.  Note  that  control  of  the  FDR  is  a less  conservative  type  of  error  control  than  that  of  the  FWE. 
Simulations  have  shown  that  “the  FDR  is  controlled  at  level  a for  the  dependent  tests  involved  in 
pairwise  comparisons  as  well  as  for  independent  tests”  (Shaffer,  1994). 

The  Benjamini  and  Hochberg  application  of  the  False  Discovery  Rate  (FDR)  criterion  can  be 
described  as  follows.  Let  q be  the  number  of  significance  tests  made  and  let 

Pi  <P2  <..  . <P^  be  the  ordered  significance  levels  of  the  q tests,  from  lowest  to  highest  probability.  Let 
a be  the  combined  significance  level  desired,  usually  0.05.  The  procedure  will  compare  F,  with  a, 

P,.i  with  a{q-\)lq,  . . .,  Pj  with  ajiq,  stopping  the  comparisons  with  the  first  j such  that  Pj  <ajlq.  All  tests 
associated  with  Pi,  . . .,  Pj  are  declared  significant;  all  tests  associated  with  Fy+,,  . . .,  F,  are  declared 
nonsignificant. 


18.5.3  Linear  and  Quadratic  Tests  of  Trends 

Tests  of  significance  designed  to  identify  consistent  patterns  of  trend  data  are  available  and, 
although  they  are  more  complex,  they  provide  more  power  to  identify  those  specific  patterns  than  a series 
of  t-  or  z-tests  would  provide. 

One  such  set  of  tests  of  significance  is  the  test  of  linear  and  test  of  quadratic  trends  applied  to  the 
long-term  trend  data  for  the  nation  and  selected  subpopulations.  The  purpose  of  this  first  set  of  general 
tests  was  to  determine  whether  the  results  of  the  series  of  assessments  in  a given  subject  could  be 
generally  characterized  as  increasing  or  decreasing,  and  whether  the  results  have  steadily  increased  (or 
decreased)  over  the  time  period  of  interest.  Simple  curvilinear  (i.e.,  quadratic)  relationships  capture  more 
complex  patterns.  For  example,  one  possible  pattern  is  to  have  initial  score  declines  over  part  of  the  time 
period  followed  by  score  increases  in  more  recent  assessments.  Another  possible  pattern  is  to  have  a 
sequence  of  several  assessments  in  which  scores  increased  followed  by  a period  of  relative  stable 
performance.  These  examples  are  two,  but  not  all,  of  the  simple  curvilinear  relationships  that  were  tested. 

The  linear  and  quadratic  components  of  the  trend  in  average  scale  scores  for  a given  subject  area 
and  age  group  were  estimated  by  applying  two  sets  of  contrasts  to  the  set  of  average  scale  scores  by  year. 
The  linear  component  of  the  trend  was  estimated  by  the  sum  i>/=2cyXy,  where  the  XjSxe.  the  average  scale 
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scores  by  year  and  the  cj  are  defined  such  that  bj  corresponds  to  the  slope  of  an  unweighted  regression  of 
the  average  scale  scores  on  the  assessment  year.  In  other  words, 

yi-jj'Ly 

k i 


where  yj  represents  an  assessment  year.  The  quadratic  component  was  estimated  by  the  sum  b2  =I}ijXj  in 
which  the  dj  are  formally  orthogonal  to  the  Cj  and  are  defined  such  that  b2  is  the  quadratic  term  in  the 
unweighted  regression  of  the  average  scale  scores  on  the  assessment  year  and  the  square  of  the 
assessment  year.  In  other  words, 


dj  = 


where 


Both  Cj  and  dj  match  expected  linear  quadratic  contrasts  in  common  texts  when  the  years  are  equally 
spaced  through  time  (Winer,  1962/1971).  The  statistical  significance  of  b,  and  b2  was  evaluated  by 
comparing  each  estimate  to  its  estimated  standard  error.  The  standard  error  of  bj  was  estimated  as  the 
square  root  of  the  sum  estimated  standard  error  of  Xj.  The  estimated 

standard  error  of  the  was  analogously  defined. 

The  linear  and  quadratic  trend  tests  allow  statements  to  be  made  about  results  across  assessment 
years  in  a more  powerful  way  than  is  possible  if  results  for  each  year  had  been  compared  to  those  of 
every  other  year,  using  a multiple-comparison  procedure  such  as  the  Bonferroni  method.  These  tests  do 
not  control  the  overall  Type  I error  rate  when  they  are  applied  to  several  related  subgroups,  such  as  the 
students  in  each  region  of  the  country.  For  this  reason,  the  Bonferroni  method  for  controlling  Type  I error 
was  used  when  the  trends  for  related  subgroups  were  tested.  For  example,  when  tests  were  conducted  for 
linear  trend  for  the  separate  race/ethnicity  groups  (i.e..  White,  Black,  and  Hispanic),  these  tests  were 
treated  as  a single  family  of  comparisons  of  size  3.  The  significance  level  for  each  of  the  separate  tests 
was  adjusted  by  the  Bonferroni  procedure  to  yield  a family-wise  error  rate  of  .05. 


18.5.4  Comparing  Proportions  Within  a Group 

Certain  analyses  involved  the  comparison  of  proportions.  One  example  was  the  comparison  of 
the  proportion  of  students  who  reported  that  a parent  graduated  from  college  to  the  proportion  of  students 
who  indicated  that  their  parents  did  not  finish  high  school  to  determine  which  proportion  was  larger. 
There  are  other  such  proportions  of  interest  in  this  example,  such  as  the  proportion  of  students  with  at 
least  one  parent  graduating  from  high  school  but  neither  parent  graduating  from  college.  For  these  types 
of  analyses,  NAEP  staff  determined  that  the  dependencies  in  the  data  could  not  be  ignored. 


\ 


408 


375 


Unlike  the  case  for  analyses  of  the  type  described  in  Section  18.5.1,  the  correlation  between  the 
proportion  of  students  reporting  a parent  graduated  from  college  and  the  proportion  reporting  that  their 
parents  did  not  finish  high  school  is  likely  to  be  negative  and  large.  For  a particular  sample  of  students,  it 
IS  likely  that  the  higher  the  proportion  of  students  reporting  “at  least  one  parent  graduated  from  college” 
IS,  the  lower  the  proportion  of  students  reporting  “neither  parent  graduated  from  high  school”  will  be.  A 
negative  dependence  will  result  in  underestimates  of  the  standard  error  if  the  estimation  is  based  on 
independence  assumptions  (as  is  the  case  for  the  procedures  described  in  Section  18.5.1).  Such 
underestimation  can  result  in  an  unacceptably  large  number  of  “nonsignificant”  differences  being 
identified  as  significant. 

The  procedures  of  Section  18.5. 1 were  modified  for  analyses  that  involved  comparisons  of 
proportions  within  a group.  The  modification  involved  using  a jackknife  method  for  obtaining  the 
standard  error  of  the  difference  in  dependent  proportions.  The  standard  error  of  the  difference  in 
proportions  was  obtained  by  first  obtaining  a separate  estimate  of  the  difference  in  question  for  each 
jackknife  replicate,  using  the  first  plausible  value  only,  then  taking  the  standard  deviation  of  the  set  of 
replicate  estimates  as  the  estimate.  The  procedures  used  for  proportions  within  a group  differed  from  the 
procedures  of  Section  18.5.1  only  with  respect  to  estimating  the  standard  error  of  the  difference;  all  other 
aspects  of  the  procedures  were  identical. 
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Chapter  19 


STATISTICAL  SUMMARY  OF  THE  1996  NAEP  SAMPLES^ 

Bruce  A.  Kaplan 
Educational  Testing  Service 


19.1  INTRODUCTION 

The  analysis  of  the  1996  NAEP  data  has  resulted  in  the  production  of  thousands  of  tables 
presenting  estimates  of  the  proficiency  of  students,  and  various  subgroups  of  students,  in  American 
schools.  This  chapter  provides  a,  statistical  summary  of  the  1996  NAEP  national  samples.  The  chapter 
assumes  a general  familiarity  with  the  structure  of  NAEP  as  summarized  in  the  Introduction  and  in  the 
overviews  presented  in  Chapters  1 and  9.  Similar  results  for  the  state  samples  appear  in  the  data 
compendia  for  the  state  mathematics  and  science  assessments. 

Two  of  the  many  types  of  NAEP  results  are  presented  here: 

1 . the  results  of  the  instrument  development  process,  including  the  sizes  of 
the  item  pools  and  numbers  of  booklets;  and 

2.  the  results  of  the  sampling  process,  including  the  numbers  of  students  in 
each  sample  by  selected  subgroups. 


19.2  MEASUREMENT  INSTRUMENTS 

For  the  1996  assessment,  79  different  assessment  booklets  and  questionnaires  were  printed  for 
age  class  9,  80  for  age  class  13,  and  81  for  age  class  17.  These  instruments  are  shown  by  age  level  and 
type  in  Table  19-1. 

The  item  pool  contributing  to  all  main  and  long-term  trend  booklets  is  described  in  Table  19-2.  In 
general,  there  are  two  types  of  items,  cognitive  and  noncognitive.  The  cognitive  items  are  developed  to 
measure  proficiency  in  particular  subject  areas,  such  as  reading  and  mathematics.  Cognitive  items  may  be 
constructed-response  or  multiple-choice.  The  noncognitive  items  are  usually  questions  about  the 
student’s  or  teacher’s  backgrounds  and  perceptions  but  may  also  probe  other  areas,  such  as  school 
policies  or  teaching  methods.  Because  many  items  were  used  at  more  than  one  age  class,  the  total  number 
of  items  in  an  item  pool  is  not  the  sum  of  the  item  pools  used  for  the  three  age  classes.  However,  results 
for  cognitive  items  that  were  common  across  two  or  three  age  classes  were  not  compared,  due  to  a NAEP 
policy  of  within  grade  scaling. 

The  SD/LEP  Student  Questionnaires,  Teacher  Questionnaires,  and  School  Characteristics  and 
Policies  Questionnaires  contained  only  noncognitive  questions.  The  number  of  items  in  the  noncognitive 


' Bruce  A.  Kaplan  was  responsible  for  the  text,  specifying  the  tables,  and  coordinating  table  production.  Shuyi  Hua  produced 
most  of  the  tables  in  this  chapter.  David  Freund’s  advice  was  invaluable  in  the  production  of  this  chapter. 
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pools  is  the  same  as  the  number  of  items  in  the  questionnaires.  More  information  about  the  instruments 
that  were  developed  is  provided  in  Chapters  2 and  4. 


19.3  SAMPLE  CHARACTERISTICS 

In  this  section,  the  characteristics  of  the  final  reporting  NAEP  samples  are  described.  The 
process  by  which  the  samples  were  selected  is  discussed  in  Chapter  3. 

In  the  1996  main  assessment,  NAEP  contacted  2,267  schools  (2,263  original  and  4 
replacements),  of  which  1,791  contributed  data  to  the  assessment.  The  disposition  of  these  schools  is 
shown  in  Table  19-3.  Some  of  the  schools  were  unwilling  to  cooperate;  others  were  believed  to  be 
eligible  from  the  sampling  frame,  but  were  not.  The  cooperation  rate  is  calculated  as  the  sum  of 
cooperating  schools  and  the  schools  that  were  found  to  have  no  eligible  students  divided  by  the  same  sum 
plus  the  schools  that  refused  or  were  from  districts  that  refused  to  cooperate. 

Table  19-3  also  shows  the  number  of  schools  in  several  categories:  region  of  the  country 
(Northeast,  Southeast,  Central,  West),  school  type  (public,  nonpublic.  Catholic,  Bureau  of  Indian  Affairs, 
Department  of  Defense  Education  Activity),  type  of  location,  number  of  teachers,  and  number  of 
students. 

For  the  1996  long-term  trend  studies,  NAEP  contacted  856  schools  (844  original  and  12 
replacements),  of  which  681  contributed  data  to  the  various  trend  assessments.  Table  19-4  supplies  the 
same  information  for  the  schools  assessed  for  the  long-term  trend  studies  that  Table  19-3  supplies  for  the 
main  assessment  schools. 

The  numbers  of  respondents  to  the  teacher  questionnaires  are  summarized  in  Table  19-5.  The 
first  column  in  this  table  includes  the  number  of  teachers  who  responded,  by  grade  and  subject  area.  The 
second  column  is  the  number  of  students  who  were  not  linked  to  teachers.  The  third  column  is  the 
number  of  students  linked  to  teachers,  but  not  specific  classes  of  these  teachers  (for  eighth  grade)  or 
teachers  who  did  not  answer  classroom  information  (for  fourth  grade).  The  last  column  is  the  number  of 
students  linked  to  their  teachers  and  their  specific  classes. 

NAEP  is  administered  in  units  called  assessment  sessions.  If  the  number  of  students  attending  an 
assessment  session  is  fewer  than  a predetermined  number,  the  students  missing  from  the  session  are 
assigned  to  a makeup  session  and  then  assessed.  Table  19-6  shows  the  number  of  regular  and  makeup 
sessions  in  1996  NAEP  by  age  class  for  the  main  and  long-term  trend  samples.  Altogether,  103,814 
assessed  and  excluded  students  were  involved  in  the  1996  NAEP. 


Tables  19-7  through  19-9  display  the  distribution  of  the  students  assessed  in  the  cross-sectional 
NAEP  assessment  in  several  basic  categories:  gender,  racial/ethnic  grouping,  region  of  the  country, 
parental  education,  type  of  location,  school  type,  and  modal  age.  These  data  are  presented  for  assessed 
students  in  the  mathematics  main  and  estimation  samples  in  Table  19-7,  the  mathematics  theme  and 
advanced  samples  in  Table  19-8,  and  the  science  main  and  advanced  samples  in  Table  19-9.  Tables  19- 
10,  19-11,  and  19-12  provide  equivalent  information,  respectively,  for  excluded  students. 


Tables  19-13,  19-14,  and  19-15  display  the  distribution  of  students  assessed  in  the  long-term 
trend  reading  and  writing  assessment  for  several  basic  categories:  gender,  racial/ethnic  grouping,  region 
of  the  country,  parental  education,  type  of  location,  and  school  type. 
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There  is  one  table  for  each  age/grade.  The  tables  have  four  columns: 

• eligible  by  age,  which  means  that  the  students  were  in  an  appropriate  age  group; 

• eligible  by  grade,  which  means  that  the  students  were  in  an  appropriate  grade; 

• eligible  by  age  and  by  grade,  which  means  that  the  students  were  of  both  an 
appropriate  age  and  appropriate  grade;  and 

• eligible  by  age  or  by  grade,  which  is  the  total  number  of  students  for  whom  data 
were  collected. 

Tables  19-16,  19-17,  and  19-18  provide  similar  information  for  the  long-term  trend  science  and 
mathematics  assessment.  Note  that  since  these  are  age-only  samples,  the  number  of  students  who  are 
age-eligible  only  will  be  the  same  as  the  number  of  students  who  are  age-  or  grade-eligible.  Likewise, 
the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the  number  of  students  who  are 
both  age-  and  grade-eligible.  Tables  19-19  through  19-24  enumerate  the  excluded  students  across  the 
various  long-term  trend  samples. 


19.4  POPULATION  ESTIMATES 

The  1996  NAEP  samples  were  designed  for  estimating  the  size  and  attributes  of  a number  of 
different  populations  of  students.  The  estimation  procedures  use  sampling  weights,  developed  by  Westat, 
Inc.,  that  are  associated  with  the  members  of  the  sample  (see  Chapter  3).  In  this  chapter,  all  estimates  of 
population  parameters  are  calculated  using  these  sampling  weights.  Note  these  estimates  are  for  the 
reporting  samples  (see  Chapter  3 for  an  explanation  of  the  reporting  and  modular  samples). 

The  sum  of  the  initial  weights  for  a given  sample  is  an  estimate  of  the  number  of  students  who 
are  in  the  population  represented  by  the  sample.  In  other  words,  the  sum  of  the  initial  weights  is  taken  as 
the  estimated  population  size.  In  analyses,  however,  this  sum  of  weights  was  rescaled  to  sum  to  the 
sample  size.  For  example,  in  Table  19-25,  the  estimated  number  of  fourth  graders  in  the  nation  is 
3,711,786,  as  estimated  from  the  main  mathematics  sample,  as  opposed  to  the  6,627  students  in  the 
sample  given  in  Table  19-7. 

Due  to  design  considerations  the  main  assessment  was  divided  into  subsamples,  and  were 
administered,  and  therefore  weighted,  independently,  so  that  the  sum  of  the  initial  weights  for  each 
subsample  estimates  the  population  size.  The  subsamples  for  mathematics  were  main,  estimation,  theme 
and  advanced;  for  science,  the  subsamples  were  main  and  advanced. 

Note  that  the  samples  for  the  main  (cross-sectional)  assessment  are  grade-only  samples,  while 
reading  and  writing  long-term  trend  are  grade  and  age  samples.  The  samples  for  the  mathematics  and 
science  long-term  trend  are  age-only  samples.  The  sum  of  the  initial  weights  of  the  excluded  students 
estimates  the  number  of  ineligible  students  at  the  respective  age/grade  levels. 


In  most  cases,  the  number  of  students  in  an  age/grade  combination  is  not  of  interest;  a researcher 
will  be  interested  in  estimating  the  number  of  students  at  either  a grade  or  an  age  level.  For  the  samples 
that  contain  both  grade-  and  age-eligible  students,  an  estimate  of  the  total  number  of  students  at  an  age 
level  can  be  made  by  summing  the  initial  weights  of  only  the  age-eligible  students  and  adding  the 
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corresponding  sample  of  age-eligible  excluded  students’  initial  weights.  An  estimate  of  the  total  number 
of  smdents  in  a grade  sample  can  be  made,  by  summing  the  initial  weights  of  grade-eligible  students  plus 
the  initial  weights  of  grade-eligible  students  from  the  appropriate  excluded  student  sample. 

Tables  19-25  to  19-42  show  the  sizes  of  the  estimated  populations  of  assessable  students  and  the 
weighted  percentages  for  the  NAEP  reporting  categories  of  gender,  race/ethnicity,  region  of  the  country, 
parents’  education  level,  type  of  location,  school  type  and  modal  age.  The  estimated  subpopulation  • . 
percentages  for  the  cross-sectional  samples  are  shown  in  Tables  19-25  through  19-30.  Tables  19-31  to  19- 
36  show  the  same  information  for  the  long-term  trend  samples.  In  a similar  manner.  Tables  19-37  to  19-42 
show  the  estimated  total  population  of  excluded  students  and  the  weighted  percentages  by  demographic 
subgroups  (data  about  parents’  education  level  is  not  collected  for  excluded  students  and  therefore  not 
reported;  data  about  reasons  for  exclusion  are  included  instead). 

In  previous  years,  this  chapter  also  provided  several  tables  showing  selected  proficiency  results  for 
assessed  students,  as  an  aid  to  readers  who  are  interested  in  the  estimates  of  proficiency  that  led  to  the 
interpretive  results  provided  in  the  NAEP  subject  area  reports.  These  tables  are  no  longer  included  in  this 
report.  Instead,  readers  are  encouraged  to  take  advantage  of  the  electronic  version  of  these  results,  in  the 
form  of  thousands  of  summary  data  tables  computed  to  analyze  the  1996  data.  The  summary  data  tables 
are  available  both  on  CD-ROM  and  via  the  World-Wide  Web  at  http://nces.ed.gov/naep. 


Table  19-1 

Measurement  Instruments  Developed  for  1996  NAEP 


Student  Assessment  Booklets 

9 

Age  Class 
13 

17 

Total  Number  of  Cross-Sectional  (MAIN) 

66' 

67' 

70 

Mathematics 

29' 

30' 

30 

Main 

26' 

26' 

26 

Estimation 

1 

1 

2 

1 

2 

Theme 

2 

Advanced^ 

1 

37 

1 

40 

Science 

37 

Main 

Advanced^ 

37 

37 

37 

3 

Total  Number  of  Long-Term  Trend 

9 

9 

8 

6 

Reading  and  Writing 

. 6 

6 

Mathematics  and  Science 

3 

3 

.2 

Total  Number  of  Questionnaires 

“ 4 

4 

3 

Excluded  Students  (Long-Term  Trend  only) 

1 

1 

1 

1 

0 

1 

SD/LEP  (Cross-Sectional  (main)  only) 

1 

1 

Teacher 

1 

1 

1 

School 

1 

Total  Number  of  Assessment  Instruments 

. 

79' 

80' 

81 

bilingual  book  was  also  used,  but  not  counted  as  a separate  book  for  this  table. 
^ No  advanced  mathematics  booklets  were  administered  to  age/class  9. 

No  advanced  science  booklets  were  administered  to  age/class  9 or  13. 


ERIC 


413 


Table  19-2 

Number  of  Items  Administered,  by  Sample  and  Age  Class 


COMMON  BACKGROUND 
Cross-Sectional  (Main  Math) 
Cross-Sectional  (Main  Science) 
Reading  and  Writing  Long-Term  Trend 
Math  and  Science  Long-Term  Trend 

MATH  MAIN 
Background 
Cognitive  — Main 
Cognitive  — Estimation 
Cognitive  ^ — Theme 
Cognitive  — Advanced 
Motivation 

SCIENCE  MAIN 
Background 
Cognitive  — Main 
Cognitive  — Advanced 
Motivation 

LONG-TERM  TREND 
Reading  Background 
Reading  Cognitive 
Writing  Background 
Writing  Cognitive 
Mathematics  Background 
Mathematics  Cognitive 
Science  Background 
Science  Cognitive 

SD/LEP  STUDENT  QUESTIONNAIRE 

MATH  TEACHER  QUESTIONNAIRE 
Teacher  Background 
Math  Background 
Math  Classroom 

SCIENCE  TEACHER  QUESTIONNAIRE 
Teacher  Background 
Science  Background 
Science  Classroom 

SCHOOL  QUESTIONNAIRE 


Distinct 

Items 

9 

13 

17 

24 

26 

36 

42 

24 

26 

36 

45 

37 

34 

48 

48 

28 

30 

48  . 

58 

25 

31 

44 

56 

144 

178 

183 

358 

31 

32 

38 

76 

14 

22 

18 

45 

0 

22 

22 

44 

5 

5 

5 

5 

39 

42 

53 

68 

141 

194 

190 

439 

0 

0 

66 

66 

5 

5 

5 

5 

40 

42 

78 

81 

105 

108 

96 

193 

53 

65 

65 

77 

6 

6 

6 

12 

3 

29 

39 

49 

68 

96 

94 

184 

16 

29 

29 

45 

63 

83 

82 

180 

114 

114 

114 

58 

77 

0 

59 

79 

17 

17 

22 

36 

40 

49 

57 

96 

77 

59 

0 

79 

14 

13 

0 

14 

59 

59 

0 

59 

100 

105 

127 

196 

ERIC 


414 


381 


Table  19-3 

School  Characteristics  in  Main  Samples  (AH. Samples) 


Grade  4 

Grade  8 

Grade  12 

Total 

TOTAL  ORIGINAL  SAMPLE 

723 

761 

779 

2,263 

Cooperating 

604 

592 

591 

1,787 

No  Eligibles  Enrolled 

20 

42 

28 

90 

School  Refused 

99 

127 

160 

386 

COOPERATION  RATE 

86 

83 

79 

83 

COOPERATING  REPLACEMENTS 

1 

1 

2 

4 

FOR  REFUSALS 
TOTALS 

Cooperating  Schools 

605 

593' 

593' 

1,791 

Completed  Questionnaires 

605 

594' 

595' 

1,794 

REGION 

Northeast 

130 

126 

123 

379 

Southeast 

134 

133 

151 

418 

Central 

165 

163 

145 

473 

West 

176 

170 

174 

520 

' Occasionally  schools  with  a completed  questionnaire  had  no  eligible  students,  so  they  were  not  included  as  participating 
cooperating  schools. 


O "32 

ERIC 
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Table  19-3  (continued) 

School  Characteristics  in  Main  Samples  (All  Samples) 


Grade  4 

Grade  8 

Grade  12 

Total 

SCHOOL  TYPE 
Public 

387 

335 

428 

1,150 

Nonpublic 

200 

243 

151 

594 

Private 

81 

91 

90 

262 

Catholic 

119 

152 

61 

332 

BIA 

0 

0 

1 

1 

DoDea 

0 

0 

0 

NUMBER  OF  TEACHERS 

Unclassified 

0 

0 

0 

0 

1-4 

15 

9 

4 

28 

5-9 

84 

61 

18 

163 

10-19 

141 

149 

52 

342 

20-49 

311 

244 

202 

757 

50-74 

30 

89 

122 

241 

75-99 

5 

20 

69 

94 

100-1- 

1 

8 

113 

122 

Missing 

18 

14 

15 

47 

NUMBER  OF  STUDENTS 

Unclassified 

0 

0 

0 

0 

1-99 

34 

18 

22 

74 

100-299 

176 

184 

103 

463 

300-499 

173 

113 

69 

355 

500-749 

133 

105 

89 

327 

750-999 

53 

86 

55 

194 

1000-1499 

12 

51 

89 

152 

1500-1- 

6 

23 

153 

182 

Missing 

18 

14 

15 

47 

Er|c 


416 


383 


Table  19-4 

School  Characteristics  in  Long-Term  Trend  Samples 


Age  Class 


9 13  17  TOTAL 


TOTAL  ORIGINAL  SAMPLE 

291 

316 

237 

844 

Cooperating 

240 

238 

191 

669 

No  Eligibles  Enrolled 

8 

27  • 

2 

37 

School  Refused 

43 

51 

44 

138 

COOPERATION  RATE 

85 

84 

81 

81 

COOPERATING  REPLACEMENTS 

FOR  REFUSALS 

8 

4 

0 

12 

TOTALS 

Cooperating  Schools 

,248 

242 

191 

681 

Completed  Questionnaires 

248 

242 

191 

681 

REGION 

Northeast 

51 

54 

36 

141 

Southeast 

62 

64 

56 

182 

Central 

59 

54 

42 

155 

West 

76 

70 

57 

203 

Er|c’“ 


417 


Table  19-5 

Numbers  of  Responses  to  Teacher  Questionnaires  and  Students  Matched  with  Teacher  Data 


Number  of 
Teachers 
Responding 

Number  of  Students  with  

No  Partial  Complete 

Match  Match  Match 

MATH 

GRADE  4 

Main 

752 

408 

99 

6,105 

Estimation 

320 

154 

11 

1,841 

Theme 

608 

351 

34 

3,405 

Advanced 

GRADES 

0 

0 

0 

0 

Main 

607 

953 

49 

6,144 

Estimation 

242 

274 

8 

1,901 

Theme 

437 

603 

51 

3,373 

Advanced 

330 

343 

9 

1,985 

GRADE  12 

Main 

0 

0 

0 

0 

Estimation 

■ 0 

0 

0 

0 

Theme 

0 

0 

0 

0 

Advanced 

SCIENCE 

404 

393 

241 

2,331 

GRADE  4 

Main 

535 

646 

159 

6,500 

Advanced 

0 

0 

0 

0 

GRADE  8 

Main 

371 

1,258 

112 

6,404 

Advanced 

0 

0 

0 

0 

GRADE  12 

Main 

0 

0 

0 

0 

Advanced 

0 

0 

0 

0 

Table  19-6 

Number  of  Students  Assessed  and  Excluded  by  Sample  and  Age  Class 


9 

— Age  Class  — 
13 

17 

ASSESSED  STUDENTS 

30,178 

34,618 

33,629  . 

Cross-Sectional 

19,745 

23,467 

25,421 

Math 

12440 

15,693 

15,453 

Main 

6,627 

7,146 

6,904 

Estimation 

2,023 

2,183 

1,849 

Theme 

3,790 

4,027 

3,735 

Advanced 

0 

2,337 

2,965 

Science 

7305 

7,774. 

9,968 

Main 

7305 

7,774 

7,537 

Advanced 

0 

0 

2,431 

Long-Term  Trend 

10,433 

11,151 

8,208 

Reading  and  Writing 

5,019 

5,493 

4,669 

Math  and  Science 

5,414 

5,658 

3,539 

EXCLUDED  STUDENTS 

2,256 

1,698 

1,435 

Cross-Sectional 

1,139 

765 

722 

Math 

383 

339 

297 

Main 

204 

166 

116 

Estimation 

43 

56 

75 

Theme 

136 

113 

99 

Advanced 

0 

4 

7 

Science 

756 

426 

425 

Main 

756 

426 

425 

Advanced 

0 

0 

0 

Long-Term  Trend 

1,117 

933 

713 

Reading  and  Writing 

532 

481 

412 

Math  and  Science 

585 

452 

301 

O 86 

ERIC 
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Table  19-7 

Number  of  Students  in  the  Mathematics  Main  and  Estimation  Samples 
by  Subgroup  Classification^  Grades  4,  S,  and  12 


MAIN  ESTIMATION 

Grade  4 Grade  8 Grade  12  Grade  4 Grade  8 Grade  12 

6,627  7,146  6,904 


TOTAL 

GENDER 

Male 

Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Asian  American 
American  Indian 
Unclassified 

REGION 

Northeast 

Southeast 

Central 

West 

PARENT’S  EDUCATION 
Less  Than  High  School 
High  School 

Greater  Than  High  School 
Graduated  College 
Unknown 

TYPE  OF  LOCATION 
Central  City 

Urban  Fringe/Large  Town 
Rural/Small  Town 

SCHOOL  TYPE 
Public 
Nonpublic 
Private 
Catholic 
BIA 
DoDEA 

MODAL  AGE 
Younger 
At  Modal  Age 
Older 


3,290 

3,597 

3,244 

3,337 

3,549 

3,660 

4,125 

4,501 

4,596 

1,106 

1,193 

1,106 

974 

911 

732 

250 

408 

339 

149 

110 

115 

23 

23 

16 

1,414 

1,312 

1,414 

1,669 

1,883 

1,924 

1,606 

1,726 

1,675 

1,938 

2,225 

1,891 

219 

466 

462 

837 

1,503 

1,300 

462 

1,310 

1,741 

2,804 

3,112 

3,177 

2,232 

736 

200 

2,380 

3,218 

2,555 

2,794 

2,186 

2,428 

1,453 

1,742 

1,921 

5,215 

5,590 

5,398 

1,412 

1,556 

1,455 

458 

576 

521 

954 

980 

934 

0 

0 

51 

0 

0 

0 

35 

48 

92 

4,197 

4,380 

4,441 

2,395 

2,718 

2,371 

2,023 

2,183 

1,849 

994 

1,052 

898 

1,029 

1,131 

951 

1,193 

1,407 

1,258 

348 

370 

273 

328 

247 

229 

98 

124 

72 

53 

26 

10 

3 

9 

7 

471 

489 

297 

540 

520 

509 

396 

549 

470 

616 

625 

573 

77 

113 

125 

227 

438 

305 

146 

361 

390 

852 

994 

985 

681 

247 

39 

859 

988 

823 

721 

698 

618 

443 

497 

408 

1,528 

1,707 

1,340 

495 

476 

509 

164 

117 

200 

331 

359 

309 

0 

0 

0 

0 

0 

0 

12 

4 

21 

1,335 

1,333 

1,194 

676 

846 

634 

ERIC 
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Table  19-8 

Number  of  Students  in  the  Mathematics  Theme  and  Advanced  Samples 
by  Subgroup  Classification,  Grades  4,  8 and  12 


Grade  4 

THEME 
Grade  8 

Grade  12 

ADVANCED 
Grade  4*  Grade  8 

Grade  12 

TOTAL 

3,790 

4,027 

3,735 

0 

2,337 

2,971 

GENDER 

Male 

1,905 

2,030 

1,797 

0 

1,130 

1,532 

Female 

1,885 

1,997 

1,938 

0 

1,207 

1,439 

RACE/ETHNICITY 

White 

2,206 

2,440 

2,279 

0 

1,650 

2,001 

Black 

655 

731 

695 

0 

280 

319 

Hispanic 

672 

641 

497 

0 

216 

327 

Asian  American 

169 

140 

228 

0 

149 

306 

American  Indian 

84 

65 

26 

0 

33 

12 

Unclassified 

4 

10 

10 

0 

9 

6 

REGION 

Northeast 

723 

608 

851 

0 

413 

638 

Southeast 

1,037 

1,125 

1,025 

0 

552 

800 

Central 

887 

937 

742 

0 

626 

666 

West 

1,143 

1,357 

1,117 

0 

746 

867 

PARENT’S  EDUCATION 

Less  Than  High  School 

162 

320 

296 

0 

84 

129 

High  School 

498 

922 

746 

0 

308 

398 

Greater  Than  High  School 

277 

732 

980 

0 

445 

664 

Graduated  College 

1,494 

1,648 

1,578 

0 

1,352 

1,709 

Unknown 

1,354 

396 

125 

0 

121 

54 

TYPE  OF  LOCATION 

Central  City 

1,721 

1,495 

1,452 

0 

968 

1,106 

Urban  Fringe/Large  Town 

1,289 

1,798 

1,339 

0 

902 

1,106 

Rural/Small  Town 

780 

734 

944 

0 

467 

759 

SCHOOL  TYPE 
Public 

3,034 

3,438 

3,075 

0 

1,661 

2,130 

Nonpublic 

756 

589 

660 

0 

676 

841 

Private 

299 

219 

235 

0 

234 

346 

Catholic 

457 

370 

425 

0 

442 

495 

BIA 

0 

0 

0 

0 

0 

0 

DoDEA 

0 

0 

0 

0 

0 

0 

MODAL  AGE 

Younger 

16 

21 

63 

0 

18 

51 

At  Modal  Age 

2,467 

2,417 

2,427 

0 

1,594 

2,113 

Older 

1,307 

1,589 

1,245 

0 

725 

807 

* Advanced  students  not  sampled  for  Grade  4. 


O .88 

ERIC 
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Table  19-9 

Number  of  Students  in  the  Science  Main  and  Advanced 
Samples  by  Subgroup  Classification,  Grades  4,  8,  and  12 


Grade  4 

MAIN 
Grade  8 

Grade  12 

ADVANCED 
Grade  4*  Grade  8* 

Grade  12 

TOTAL 

7,305 

7,774 

7,537 

0 

0 

2,431 

GENDER 

Male 

3,651 

3,872 

3,547 

0 

0 

1,167 

Female 

3,654 

3,902 

3,990 

0 

0 

1,264 

RACE/ETHNICITY 

White 

4,106 

4,292 

4,748 

0 

0 

1,714 

Black 

1,251 

1,492 

1,225 

0 

0 

293 

Hispanic 

1,352 

1,426 

1,015 

0 

0 

197 

Asian  American 

356 

382 

458 

0 

0 

209 

American  Indian 

223 

149 

70 

0 

0 

12 

Unclassified 

17 

33 

21 

0 

0 

6 

REGION 

Northeast 

1,503 

1,068 

1,562 

0 

0 

541 

Southeast 

1,843 

2,246 

2,148 

0 

0 

695 

Central 

1,699 

1,595 

1,589 

0 

0 

634 

West 

2,260 

2,865 

2,238 

0 

0 

561 

PARENT’S  EDUCATION 

Less  Than  High  School 

271 

553 

606 

0 

0 

87 

High  School 

938 

1,471 

1,414 

0 

0 

272 

Greater  Than  High  School 

544 

1,428 

1,879 

0 . 

0 

526 

Graduated  College 

2,994 

3,400 

3,308 

0 

0 

1,476 

Unknown 

2,433 

774 

211 

0 

0 

0 

TYPE  OF  LOCATION 

Central  City 

3,228 

3,055 

3,080 

0 

0 

949 

Urban  Fringe/Large  Town 

2,769 

2,963 

2,488 

0 

0 

895 

Rural/Small  Town 

1,308 

1,756 

1,969 

0 

0 

587 

SCHOOL  TYPE 
Public 

5,814 

6,376 

6,112 

0 

0 

1,739 

Nonpublic 

1,491 

1,398 

1,425 

0 

0 

692 

Private 

499 

597 

499 

0 

0 

185 

Catholic 

992 

801 

926 

0 

0 

507 

BIA 

0 

0 

0 

0 

0 

0 

DODEA 

0 

0 

0 

0 

0 

0 

MODAL  AGE 

Younger 

46 

46 

93 

0 

0 

38 

At  Modal  Age 

4,739 

4,553 

4,802 

0 

0 

1,720 

Older 

2,520 

3,175 

2,642 

0 

0 

673 

*Advanced  students  not  sampled  for  Grade  4 and  Grade  8. 


ERIC 
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Table  19-10 

Number  of  Excluded  Students  in  the  Mathematics  Main  and  Estimation 
Samples  by  Subgroup  Classification,  Grades  4,  8,  and  12 


Grade  4 

. MAIN 
Grade  8 

Grade  12 

ESTIMATION 

Grade  4 Grade  8 Grade  12 

TOTAL 

204 

166 

116 

43 

56 

75 

GENDER 

Male 

122 

104 

72 

33 

34 

44 

Female 

82 

62 

44 

10 

22 

31 

RACE/ETHNICITY 

White 

92 

100 

65 

24 

33 

32 

Black 

34 

30 

22 

7 

8 

16 

Hispanic 

66 

18 

25 

9 

12 

23 

Asian  American 

8 

10 

3 

2 

3 

4 

American  Indian 

3 

3 

1 

1 

0 

0 

Unclassified 

1 

5 

0 

0 

0 

0 

REGION 

Northeast 

21 

45 

22 

5 

19 

8 

Southeast 

49 

34 

27 

9 

11 

9 

Central 

29 

36 

18 

14 

11 

9 

West 

105 

51 

49 

15 

15 

49 

PARENT’S  EDUCATION 

Less  Than  High  School 

0 

0 

0 

0 

0 

0 

High  School 

0 

0 

0 

0 

0 

0 

Greater  Than  High  School 

0 

0 

0 

0 

0 

0 

Graduated  College 

0 

0 

0 

0 

0 

0 

Unknown 

0 

0 

0 

0 

0 

0 

TYPE  OF  LOCATION 

Central  City 

82 

64 

40 

10 

29 

36 

Urban  Fringe/Large  Town 

61 

50 

44 

21 

15 

13 

Rural/Small  Town 

61 

52 

32 

12 

12 

26 

SCHOOL  TYPE 

Public 

197 

162 

115 

43 

56 

75 

Nonpublic 

7 

4 

1 

0 

0 

0 

Private 

1 

0 

0 

0 

0 

0 

Catholic 

6 

4 

1 

0 

0 

0 

BIA 

0 

0 

0 

0 

0 

0 

DoDEA 

0 

0 

0 

0 

0 

0 

MODAL  AGE 

Younger 

0 

3 

1 

0 

0 

0 

At  Modal  Age 

106 

48 

31 

22 

21 

17 

Older 

98 

115 

84 

21 

35 

58 

O 390 

ERIC 
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Table  19-11 

Number  of  Excluded  Students  in  the  Mathematics  Theme  and  Advanced 
Sample  by  Subgroup  Classification,  Grades  4,  8,  and  12 


Grade  4 

THEME 
Grade  8 

Grade  12 

ADVANCED 
Grade  4*  Grade  8 

Grade  12 

TOTAL 

136 

II3 

99 

0 

4 

7 

GENDER 

Male 

71 

59 

55 

0 

4 

6 

Female 

65 

54 

44 

0 

0 

1 

RACE/ETHNICITY 

White 

37 

34 

55 

0 

2 

2 

Black 

26 

30 

17 

0 

0 

0 

Hispanic 

60 

44 

19 

0 

0 

0 

Asian  American 

9 

3 

7 

0 

2 

5 

American  Indian 

3 

2 

I 

0 

0 

0 

Unclassified 

I 

0 

0 

0 

0 

0 

REGION 

Northeast 

18 

15 

30 

0 

0 

2 

Southeast 

23 

37 

25 

0 

2 

0 

Central 

29 

15 

8 

0 

0 

2 

West 

66 

46 

36 

0 

2 

3 

PARENT’S  EDUCATION 

Less  Than  High  School 

0 

0 

0 

0 

0 

0 

High  School 

0 

0 

0 

0 

0 

0 

Greater  Than  High  School 

0 

0 

0 

0 

0 

0 

Graduated  College 

0 

0 

0 

0 

0 

0 

Unknown 

0 

0 

0 

0 

0 

0 

TYPE  OF  LOCATION 

Central  City 

91 

45 

52 

0 

2 

4 

Urban  Fringe/Large  Town 

35 

47 

31 

0 

0 

1 

Rural/Small  Town 

10 

21 

16 

0 

2 

2 

SCHOOL  TYPE 

Public 

133 

II2 

91 

0 

4 

7 

Nonpublic 

3 

I 

8 

0 

0 

0 

Private 

I 

0 

8 

0 

0 

0 

Catholic 

2 

I 

0 

0 

0 

0 

BIA 

0 

0 

0 

0 

0 

0 

DoDEA 

0 

0 

0 

0 

0 

0 

MODAL  AGE 

Younger 

2 

3 

I 

0 

0 

0 

At  Modal  Age 

77 

46 

21 

0 

2 

3 

Older 

57 

64 

77 

0 

2 

4 

*Advanced  students  not  sampled  for  Grade  4. 
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Table  19-12 

Number  of  Excluded  Students  in  the  Science  Main  and  Advanced  Samples 
by  Subgroup  Classification,  Grades  4 ,8,  and  12 


Grade  4 

MAIN 
Grade  8 

Grade  12 

Grade  4 

ADVANCED 
Grade  8 

Grade  12 

TOTAL 

756 

426 

425 

0 

0 

0 

SEX 

' Male 

457 

265 

259 

0 

0 

0 

Female 

299 

161 

166 

0 

0 

0 

RACE/ETHNICITY 

White 

239 

145 

185 

0 

0 

0 

Black 

124 

98 

103 

0 

0 

0 

Hispanic 

317 

159 

99 

0 

0 

0 

Asian  American 

65 

15 

33 

0 

0 

0 

American  Indian 

5 

1 

2 

0 

0 

0 

Unclassified 

6 

2 

3 

0 

0 

0 

REGION 

Northeast 

91 

38 

84 

0 

0 

0 

Southeast 

170 

119 

112 

0 

0 

0 

Central 

132 

40 

57 

0 

0 

0 

West 

363 

229 

172 

0 

0 

0 

PARENTS  EDUCATION 

Less  Than  High  School 

0 

0 

0 

0 

0 

0 

High  School 

0 

1 

0 

0 

0 

0 

Greater  Than  High  School 

0 

0 

0 

0 

0 

0 

Graduated  College 

0 

0 

0 

0 

0 

0 

Unknown 

0 

0 

0 

0 

0 

0 

TYPE  OF  LOCATION 

Central  City 

482 

187 

212 

0 

0 

0 

Urban  Fringe/Large  Town 

178 

136 

127 

0 

0 

0 

Rural/Small  Town 

96 

103 

86 

0 

0 

0 

SCHOOL  TYPE 

Public 

752 

424 

419 

0 

0 

0 

Nonpublic 

4 

2 

6 

0 

0 

0 

Private 

2 

0 

5 

0 

0 

0 

Catholic 

2 

2 

1 

0 

0 

0 

BIA 

0 

0 

0 

0 

0 

0 

DoDEA 

0 

0 

0 

0 

0 

0 

MODAL  AGE 

<Modal  Age 

8 

5 

6 

0 

0 

0 

=Modal  Age 

363 

144 

113 

0 

0 

0 

>Modal  Age 

385 

277 

306 

0 

0 

0 

O 392 

ERIC 


425 


Table  19-13 

Number  of  Students  in  the  Reading  and  Writing  Long-Term  Trend 
Sample  by  Type  of  Eligibility  and  Subgroup  Classification,  Age  9/Grade  4 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,654 

3,789 

2,424 

5,019 

GENDER 

Male 

1,808 

1,838 

1,128 

2,518 

Female 

1,846 

1,951 

1,296 

2,501 

RACE/ETHNICITY 

White 

2,067 

2,183 

1,356 

2,894 

Black 

598 

634 

421 

811 

Hispanic 

741 

727 

462 

1,006 

Asian  American 

139 

151 

126 

164 

American  Indian 

99 

83 

50 

132 

Unclassified 

10 

11 

9 

12 

REGION 

Northeast 

795 

861 

630 

1,026 

Southeast 

1,027 

1,019 

622 

1,424 

Central 

707 

747 

413 

1,041 

West 

1,125 

1,162 

759 

1,528 

PARENT’S  EDUCATION 

Less  Than  High  School 

158 

176 

99 

235 

High  School 

574 

615 

367 

822 

Greater  Than  High  School 

187 

181 

117 

251 

Graduated  College 

1,430 

1,562 

1,016 

1,976 

Unknown 

1,274 

1,212 

806 

1,680 

TYPE  OF  LOCATION 

Central  City 

1,513 

1,635 

1,019 

2,129 

Urban  Fringe/Large  Town 

1,271 

1,297 

893 

1,675 

Rural/Small  Town 

870 

857 

512 

1,215 

SCHOOL  TYPE 

Public 

3,237 

3,342 

2,116 

4,463 

Nonpublic 

417 

447 

308 

556 

Private 

174 

169 

116 

227 

Catholic 

243 

278 

192 

329 

BIA 

0 

0 

0 

0 

DoDEA 

0 

0 

0 

0 

O 

ERIC 


4Z6 


393 


Table  19-14 

Number  of  Students  in  the  Reading  and  Writing  Long-Term  Trend 
Sample  by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13/Grade  8 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,847 

4,150 

2,504 

5,493 

GENDER 

Male 

1,870 

2,060 

1,124 

2,806 

Female 

1,977 

2,090 

1,380 

2,687 

RACE/ETHNICITY 

White 

2,389 

2,550 

1,518 

3,421 

Black 

540 

593. 

348 

785 

Hispanic 

565 

635 

373 

827 

Asian  American 

222 

226 

178 

270 

American  Indian 

125 

141 

84 

182 

Unclassified 

6 

5 

3 

8 

REGION 

Northeast 

811 

894 

582 

1,123 

Southeast 

1,088 

1,159 

662 

1,585 

Central 

771 

854 

478 

1,147 

West 

1,177 

1,243 

782 

1,638 

PARENT’S  EDUCATION 

Less  Than  High  School 

217 

301 

143 

375 

High  School 

1,089 

1,172 

690 

1,571 

Greater  Than  High  School 

406 

441 

281 

566 

Graduated  College 

1,725 

1,846 

1,164 

2,407 

Unknown 

394 

375 

218 

551 

TYPE  OF  LOCATION 

Central  City 

1,441 

1,560 

937 

2,064 

Urban  Fringe/Large  Town 

1,412 

1,455 

952 

1,915 

Rural/Small  Town 

994 

1,135 

615 

1,514 

SCHOOL  TYPE 

Public 

3,421 

3,720 

2,217 

4,924 

Nonpublic 

410 

396 

' 274 

532 

Private 

190 

190 

138 

242 

Catholic 

220 

206 

136 

290 

BIA 

16 

34 

13 

37 

DODEA 

0 

0 

0 

0 

er|c 


^9A 


Table  19-15 

Number  of  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  17/Grade  11 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,681 

3,737 

2,749 

4,669 

GENDER 

Male 

1,874 

1,943 

1,356 

2,461 

Female 

1,807 

1,794 

1,393 

2,208 

RACE/ETHNICITY 

White 

2,528 

2,573 

1,986 

3,115 

Black 

449 

440 

279 

610 

Hispanic 

465 

468 

302 

631 

Asian  American 

163 

178 

123 

218 

American  Indian 

69 

67 

52 

84 

Unclassified 

7 

11 

7 

11 

REGION 

Northeast 

682 

721 

523 

880 

Southeast 

1,063 

1,038 

748 

1,353 

Central 

871 

, 900 

686 

1,085 

West 

1,065 

1,078 

792 

1,351 

PARENT’S  EDUCATION 

Less  Than  High  School 

251 

254 

151 

354 

High  School 

945 

947 

675 

1,217 

Greater  Than  High  School 

672 

692 

528 

836 

Graduated  College 

1,673 

1,717 

1,316 

2074 

Unknown 

103 

96 

61 

138 

TYPE  OF  LOCATION 

Central  City 

1,111 

1,101 

780 

1,432 

Urban  Fringe/Large  Town 

1,537 

1,577 

1,196 

1,918 

Rural/Small  Town 

1,033 

1,059 

773 

1,319 

SCHOOL  TYPE 

Public 

3,384 

3,411 

2,511 

4,284 

Nonpublic 

289 

318 

230 

377 

Private 

140 

145 

105 

180 

Catholic 

149 

173 

125 

197 

BIA 

8 

8 

8 

8 

DODEA 

0 

0 

0 

0 

O 

ERIC 


428 


395 


Table  19-16 

Number  of  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  9‘ 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

5414 

3,665 

3,665 

5,414 

GENDER 

Male 

2,709 

1,766 

1,766 

2,709 

Female 

2,705 

1,899 

1,899 

2,705 

RACE/ETHNICITY 

White 

3,204 

2,146 

2,146 

3,204 

Black 

801 

578 

578 

801 

Hispanic 

1,075 

687 

687 

. 1,075 

Asian  American 

188 

156 

156 

188 

American  Indian 

134 

88 

88 

134 

Unclassified 

12 

10 

10 

12 

REGION 

Northeast 

1,142 

918 

918 

1,142 

Southeast 

1,436 

906 

906 

1,436 

Central 

1,188 

698 

698 

1,188 

West 

1,648 

1,143 

1,143 

1,648 

PARENT’S  EDUCATION 

Less  Than  High  School 

230 

146 

146 

230 

High  School 

713 

466 

466 

713 

Greater  Than  High  School 

386 

289 

289 

386 

Graduated  College 

2,274 

1,604 

1,604 

2,274 

Unknown 

1,792 

1,150 

1,150 

1,792 

TYPE  OF  LOCATION 

Central  City 

2,485 

1,721 

1,721 

2,485 

Urban  Fringe/Large  Town 

1,670 

1,198 

1,198 

1,670 

Rural/Small  Town 

1,259 

746 

746 

1,259 

SCHOOL  TYPE 

Public 

4,790 

3,231 

3,231 

4,790 

Nonpublic 

609 

422 

422 

609 

Private 

162 

104 

104 

162 

Catholic 

447 

318 

318 

447 

BIA 

15 

12 

12 

15 

DODEA 

0 

0 

0 

0 

' Note:  Since  this  is  an  age-only  sample,  the  number  of  students  who  are  age-eligible  only  will  be  the  same  as  the  number  of 
students  who  are  age-  or  grade-eligible.  Likewise,  the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the 
number  of  students  who  are  both  age-  and  grade-eligible. 


O 16 
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Table  19-17 

Number  of  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13' 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

5,658 

3,662 

3,662 

5,658 

GENDER 

Male 

2,736 

1,652 

1,652 

2,736 

Female 

2,922 

2,010 

2,010 

2,922 

RACE/ETHNICITY 

White 

3,528 

2,272 

2,272 

3,528 

Black 

776 

509 

509 

776 

Hispanic 

943 

565 

565 

943 

Asian  American 

293 

234 

234 

293 

American  Indian 

112 

76 

76 

112 

Unclassified 

6 

6 

. 6 

6 

REGION 

Northeast 

1,221 

900 

900 

1,221 

Southeast 

1,589 

937 

937 

1,589 

Central 

1,129 

693 

693 

1,129 

West 

1,719 

1,132 

1,132 

1,719 

PARENT’S  EDUCATION 

Less  Than  High  School 

353 

188 

188 

353 

High  School 

1,295 

815 

815 

1,295 

Greater  Than  High  School 

943 

672 

672 

943 

Graduated  College 

2,458 

1,655 

1,655 

2,458 

Unknown 

587 

320 

320 

587 

TYPE  OF  LOCATION 

Central  City 

2,063 

1,357 

1,357 

2,063 

Urban  Fringe/Large  Town 

2,047 

1,386 

1,386 

2,047  . 

Rural/Small  Town 

1,548 

919 

919 

1,548 

SCHOOL  TYPE 

Public 

5,096 

3,260 

3,260 

5,096 

Nonpublic 

562 

402 

402 

562 

Private 

• 224 

181 

181 

224 

Catholic 

338 

221 

221 

338 

BIA 

0 

0 

0 

0 

DODEA 

0 

0 

0 

0 

' Note;  Since  this  is  an  age-only  sample,  the  number  of  students  who  are  age-eligible  only  will  be  the  same  as  the  number  of 
students  who  are  age-  or  grade-eligible.  Likewise,  the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the 
number  of  students  who  are  both  age-  and  grade-eligible. 
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Table  19-18 

Number  of  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  17’ 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,539 

2,532 

2,532 

3,539 

GENDER 

Male 

1,755 

1,196 

1,196 

1,755 

Female 

1,784 

1,336 

1,336 

1,784 

RACE/ETHNICITY 

White 

2,401 

1,836 

1,836 

2,401 

Black 

531 

329 

329 

531 

Hispanic 

401 

244 

244 

401 

Asian  American 

155 

94 

94 

155 

American  Indian 

43 

23 

23 

43 

Unclassified 

8 

6 

6 

8 

REGION 

Northeast 

712 

519 

519 

712 

Southeast 

1,122 

803 

803 

1,122 

Central 

733 

529 

529 

733 

West 

972 

681 

681 

972 

PARENT’S  EDUCATION 

Less  Than  High  School 

236 

122 

122 

236 

High  School 

757 

506 

506 

757 

Greater  Than  High  School 

• 835 

616 

616 

835 

Graduated  College 

1,619 

1,238 

1,238 

1,619 

Unknown 

71 

37 

37 

71 

TYPE  OF  LOCATION 

Central  City 

1,311 

896 

896 

1,311 

Urban  Fringe/Large  Town 

1,189 

883 

883 

1,189 

Rural/Small  Town 

1,039 

753 

753 

1,039 

SCHOOL  TYPE 
Public 

3,257 

2,309 

2,309 

3,257 

Nonpublic 

282 

223 

223 

282 

Private 

124 

99 

99 

124 

Catholic 

158 

124 

124 

158 

BIA 

0 

0 

0 

0 

DODEA 

0 

0 

0 

0 

' Note:  Since  this  is  an  age-only  sample,  the  number  of  students  who  are  age-eligible  only  will  be  the  same  as  the  number  of 
students  who  are  age-  or  grade-eligible.  Likewise,  the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the 
number  of  students  who  are  both  age-  and  grade-eligible. 


O ‘98 
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Table  19-19 

Number  of  Excluded  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample  by  Type  of  Eligibility 

and  Subgroup  Classification,  Age  9/Grade  4 


iHlirririlo 

Age 

uy 

Age  & 
Grade  Grade 

Age  or 
Grade 

TOTAL 

345 

404 

217 

532 

SEX 

Male 

207 

243 

124 

326 

Female 

138 

161 

93 

206 

RACE/ETHNICITY 

White 

133 

161 

66 

228 

Black 

54 

66 

30 

90 

Hispanic 

122 

134 

90 

166 

Asian  American 

30 

35 

26 

39 

American  Indian 

3 

6 

3 

6 

Unclassified 

3 

2 

2 

3 

REGION 

Northeast 

30 

46 

19 

57 

Southeast 

96 

130 

52 

174 

Central 

49 

47 

18 

78 

West 

170 

181 

128 

223 

TYPE  OF  LOCATION 

Central  City 

189 

211 

130 

270 

Urban  Fringe/Large  Town 

106 

117 

69 

154 

Rural/Small  Town 

50 

. 76 

18 

108 

SCHOOL  TYPE 

Public 

340 

402 

215 

527 

Nonpublic 

4 

2 

2 

4 

Private 

0 

0 

0 

0 

Catholic 

4 

2 

2 

4 

BIA 

0 

0 

0 

0 

DoDea 

0 

0 

0 

0 

ERIC 


4S2 


399 


Table  19-20 

Number  of  Excluded  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample  by  Type  of  Eligibility 

and  Subgroup  Classification,  Age  13/Grade  8 


Eligible  by 


Age 

Grade 

Age  & 
Grade 

Age  or 
Grade 

TOTAL 

265 

303 

87 

481 

SEX 

Male 

175 

211 

67 

319 

Female 

90 

92 

20 

162 

RACE/ETHNICITY 

White 

153 

187 

47 

293 

Black 

33 

35 

8 

60 

Hispanic 

58 

58 

23 

93 

Asian  American 

10 

11 

16 

American  Indian 

12 

4 

18 

Unclassified 

1 

0 

0 

1 

REGION 

Northeast 

45 

46 

17 

74 

Southeast 

93 

104 

21 

176 

Central 

41 

74 

14 

101 

West 

86 

79 

35 

130 

TYPE  OF  LOCATION 

Central  City 

118 

121 

35 

204 

Urban  Fringe/Large  Town 

79 

80 

35 

124 

Rural/Small  Town 

68 

102 

17 

153 

SCHOOL  TYPE 

Public 

257 

291 

84 

464 

Nonpublic 

3 

1 

0 

4 

Private 

2 

1 

0 

3 

Catholic 

1 

0 

0 

1 

BIA 

5 

11 

3 

13 

DoDea 

0 

0 

0 

0 

O 'TO 

ERIC 


4?3 


Table  19-21 

Number  of  Excluded  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample  by  Type  of  Eligibility 

and  Subgroup  Classification,  Age  17/Grade  11 


Eligible  by 


Age 

Grade 

Age  & 
Grade 

Age  or 
Grade 

TOTAL 

277 

227 

92 

412 

SEX 

Male 

171 

139 

43 

267 

Female 

106 

88 

49 

145 

RACE/ETHNICITY 

White 

161 

151 

63 

249 

Black 

65 

33 

18 

80 

Hispanic 

35 

26 

7 

54 

Asian  American 

10 

10 

1 

19 

American  Indian 

4 

7 

3 

8 

Unclassified 

2 

0 

0 

2 

REGION 

Northeast 

36 

44 

18 

62 

Southeast 

117 

75 

32 

160 

Central 

51 

41 

15 

77 

West 

73 

67 

27 

113 

TYPE  OF  LOCATION 

Central  City 

80 

68 

25 

123 

Urban  Fringe/Large  Town 

107 

105 

46 

166 

Rural/Small  Town 

90 

54 

21 

123 

SCHOOL  TYPE 

Public 

277 

224 

92 

409 

Nonpublic 

0 

1 

0 

1 

Private 

0 

0 

0 

0 

Catholic 

0 

1 

0 

1 

BIA 

0 

2 

0 

2 

DoDea 

0 

0 

0 

0 

434 


401 


Table  19-22 

Number  of  Excluded  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample  by  Type 
Eligibility  and  Subgroup  Classification,  Age  9/Grade  4 


Age 

Age  & 
Grade  Grade 

Age  or 
Grade 

TOTAL 

585 

316 

316 

585 

SEX 

Male 

360 

192 

192 

360 

Female 

225 

124 

124 

225 

RACE/ETHNICITY 

White 

220 

103 

103 

220 

Black 

96 

41 

41 

96 

Hispanic 

217 

133 

133 

217 

Asian  American 

45 

35 

35 

45 

American  Indian 

2 

1 

1 

2 

Unclassified 

5 

3 

3 

5 

REGION 

Northeast 

65 

38 

38 

65 

Southeast 

202 

72 

72 

202 

Central 

80 

35 

35 

80 

West 

238 

171 

171 

238 

TYPE  OF  LOCATION 

Central  City 

334 

200 

200 

334 

Urban  Fringe/Large  Town 

148 

91 

91 

148 

Rural/Small  Town 

103 

25 

25 

103 

SCHOOL  TYPE 

Public 

578 

314 

314 

578 

Nonpublic 

7 

2 

2 

7 

Private 

0 

0 

0 

0 

Catholic 

7 

2 

2 

7 

BIA 

0 

0 

0 

0 

DoDea 

0 

0 

0 

0 

ERIC 


Table  19-23 

Number  of  Excluded  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample  by  Type  of 
Eligibility  and  Subgroup  Classification,  Agel 3/Grade  8 


Eligible  by 


Age 

Grade 

Age  & 
Grade 

Age  or 
Grade 

TOTAL 

452 

150 

150 

452 

SEX 

Male 

286 

86 

86 

286 

Female 

166 

64 

64 

166 

RACE/ETHNICITY 

White 

239 

75 

75 

239 

Black 

76 

20 

20 

76 

Hispanic 

116 

47 

47 

116 

Asian  American 

16 

7 

7 

16 

American  Indian 

2 

1 

1 

2 

Unclassified 

3 

0 

0 

3 

REGION 

Northeast 

87 

34 

34 

87 

Southeast 

156 

41 

41 

156 

Central 

89 

25 

25 

89 

West 

120 

50 

50 

120 

TYPE  OF  LOCATION 

Central  City 

187 

57 

57 

187 

Urban  Fringe/Large  Town 

125 

57 

57 

125 

Rural/Small  Town 

140 

36 

36 

140 

SCHOOL  TYPE 

Public 

450 

150 

150 

450 

Nonpublic 

2 

0 

0 

2 

Private 

2 

0 

0 

2 

Catholic 

0 

0 

0 

0 

BIA 

0 

0 

0 

0 

DoDea 

0 

0 

0 

0 

4 3:6 
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Table  19-24 

Number  of  Excluded  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample  by  Type  of 
Eligibility  and  Subgroup  Classification,  Agel7/Grade  11 


Age 

— — Eligible  by 

Age  & 
Grade  Grade 

Age  or 
Grade 

TOTAL 

301 

110 

110 

301 

SEX 

Male 

202 

67 

67 

202 

Female 

99 

43 

43 

99 

RACE/ETHNICITY 

White 

170 

75 

75 

170 

Black 

55 

17 

17 

55 

Hispanic 

55 

14 

14 

55 

Asian  American 

16 

3 

3 

16 

American  Indian 

5 

1 

1 

5 

Unclassified 

0 

0 

0 

0 

REGION 

Northeast 

57 

36 

36 

57 

Southeast 

103 

24 

24 

103 

Central 

51 

18 

18 

51 

West 

90 

32 

32 

90 

TYPE  OF  LOCATION 

Central  City 

109 

36 

36 

• 109 

Urban  Fringe/Large  Town 

107 

50 

50 

107 

Rural/Small  Town 

85 

24 

24 

85 

SCHOOL  TYPE 

Public 

298 

110 

110 

298 

Nonpublic 

3 

0 

0 

3 

Private 

3 

0 

0 

3 

Catholic 

0 

0 

0 

0 

BIA 

0 

0 

0 

0 

DoDea 

0 

0 

0 

0 

O ^-04 

ERIC 
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Table  19-25 

Weighted  Percentage  of  Students  in  the  Mathematics  Main  and  Estimation  Samples 
by  Subgroup  Classification,  Grades  4,  8,  and  12 


MAIN  ESTIMATION 

Grade  4 Grade  8 Grade  12  Grade  4 Grade  8 Grade  12 

total  3,711,786  3,566,392  2,827,040  3,688,821  3,598,564  2,740,931 

GENDER 


Male 

Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Asian  American 
American  Indian 
Unclassified 

REGION 

Northeast 

Southeast 

Central 

West 

PARENT’S  EDUCATION 
Less  Than  High  School 
High  School 

Greater  Than  High  School 
Graduated  College 
Unknown 

TYPE  OF  LOCATION 
Central  City 

Urban  Fringe/Large  Town 
Rural/Small  Town 

SCHOOL  TYPE 
Public 
Non  Public 
Private 
Catholic 
BIA 
DoDEA 

MODAL  AGE 
Younger 
At  Modal  Age 
Older 


50.8 

52.3 

47.6 

49.2 

47.7 

52.4 

67.8 

69.1 

69.6 

14.6 

14.2 

14.1 

12.9 

12.3 

11.2 

2.7 

3.3 

3.6 

1.7 

1.1 

1.3 

0.2 

0.1 

0.2 

21.9 

20.3 

21.8 

21.0 

23.3 

21.6 

24.8 

24.3 

24.0 

32.3 

32.1 

32.6 

3.8 

6.9 

6.4 

12.5 

21.8 

18.8 

7.0 

18.5 

25.4 

39.4 

41.7 

46.4 

35.6 

10.7 

2.7 

30.1 

33.3 

31.7 

46.2 

36.2 

39.8 

23.7 

30.5 

28.5 

89.1 

89.3 

87.3 

10.9 

10.7 

12.0 

3.7 

4.5 

4.2 

7.2 

6.3 

7.9 

0.0 

0.0 

0.6 

0.0 

0.0 

0.0 

0.6 

0.6 

1.3 

59.6 

56.2 

64.9 

39.8 

43.1 

33.8 

49.3 

50.4 

47.4 

50.7 

49.6 

52.6 

68.0 

69.6 

70.3 

14.6 

14.1 

13.8 

13.0 

11.9 

11.4 

2.5 

3.3 

3.9 

1.9 

0.9 

0.4 

0.1 

0.2 

0.2 

21.6 

20.7 

23.5 

22.1 

21.3 

20.6 

24.6 

24.4 

24.5 

31.7 

33.7 

31.5 

3.8 

6.0 

6.6 

11.6 

22.0 

17.2 

7.5 

16.3 

21.0 

40.0 

42.5 

52.9 

34.9 

11.5 

2.1 

37.3 

37.1 

39.3 

36.5 

36.8 

34.1 

26.2 

26.1 

26.6 

85.4 

88.6 

83.1 

14.6 

11.4 

16.9 

5.7 

4.0 

6.9 

8.9 

7.4 

10.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.6 

0.1 

1.1 

60.8 

55.6 

65.2 

38.6 

44.3 

33.7 
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Table  19-26 

Weighted  Percentage  of  Students  in  the  Mathematics  Theme  and  Advanced  Mathematics 
Samples  by  Subgroup  Classification,  Grades  4,  8.  and  12 


THEME  ADVANCED 


Grade  4 

Grade  8 

Grade  12 

Grade  4* 

Grade  8 

Grade  12 

TOTAL 

3,690,245 

3,566,103 

2,845,023 

0 

809,085 

696,805 

GENDER 

Male 

51.8 

52.6 

49.2 

0.0 

48.3 

50.6 

Female 

48.2 

. 47.4 

50.8 

0.0 

51.7 

49.4 

RACE/ETHNICITY 

White 

68.5 

70.5 

69.3 

0.0 

71.0 

74.1 

Black 

14.5 

13.7 

14.2 

0.0 

14.3 

7.2 

Hispanic 

12.5 

11.6 

11.6 

0.0 

6.5 

7.9 

Asian  American 

2.9 

2.4 

4.2 

0.0 

5.6 

10.3 

American  Indian 

1.5 

1.7 

0.5 

0.0 

2.2 

0.3 

Unclassified 

0.1 

0.1 

0.2 

0.0 

0.4 

0.2 

REGION 

Northeast 

21.0 

20.7 

23.2 

0.0 

26.9 

24.8 

Southeast 

22.9 

22.5 

21.2 

0.0 

16.9 

21.0 

Central 

25.7 

23.0 

23.4 

0.0 

29.9 

28.6 

West 

30.4 

33.8 

32.1 

0.0 

26.3 

25.6 

PARENT’S  EDUCATION 

Less  Than  High  School 

4.5 

7.9 

6.9 

0.0 

3.0 

3.8 

High  School 

12.6 

23.8 

19.1 

0.0 

13.7 

13.3 

Greater  Than  High  School 

7.4 

18.1 

26.5 

0.0 

20.2 

22.6 

Graduated  College 

40.1 

40.5 

44.2 

0.0 

57.4 

58.5 

Unknown 

35.3 

9.4 

3.2 

0.0 

4.7 

1.5 

TYPE  OF  LOCATION 

Central  City 

39.9 

28.1 

31.6 

0.0 

36.1 

32.0 

Urban  Fringe/Large  Town 

37.7 

49.2 

41.3 

0.0 

38.7 

38.5 

Rural/Small  Town 

22.4 

22.7 

27.1 

0.0 

25.2 

29.4 

SCHOOL  TYPE 

Public 

87.6 

90.5 

88.7 

0.0 

83.5 

81.5 

Nonpublic 

12.4 

9.5 

11.3 

0.0 

16.5 

18.5 

Private 

5.0 

3.9 

3.9 

0.0 

5.3 

7.9 

Catholic 

7.4 

5.5 

7.4 

0.0 

11.3 

10.6 

BIA 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

DoDEA 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

MODAL  AGE 

Younger 

0.4 

0.6 

1.5 

0.0 

0.7 

1.5 

At  Modal  Age 

59.0 

55.5 

64.4 

0.0 

68.7 

72.2 

Older 

40.5 

43.9 

34.1 

0.0 

30.6 

26.3 

* Advanced  students  not  sampled  for  Grade  4. 
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Table  19-27 

Weighted  Percentage  of  Students  in  the  Science  Main  and  Advanced 
Samples  by  Subgroup  Classification,  Grades  4,  8,  and  12 


MAIN 

Grade  4 Grade  8 Grade  12 


ADVANCED 

Grade  4*  Grade  8‘  Grade  12 


TOTAL 

GENDER 


3,618,494  3,564,079  2,903,402 


50.3 

49.7 


50.8 

49.2 


48.4 

51.6 


0 


0 585,798 


Male 

Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Asian  American 
American  Indian 
Unclassified 

REGION 

Northeast 

Southeast 

Central 

West 

PARENT’S  EDUCATION 
Less  Than  High  School 
High  School 

Greater  Than  High  School 
Graduated  College 
Unknown 

TYPE  OF  LOCATION 
Central  City 

Urban  Fringe/Large  Town 
Rural/Small  Town 

SCHOOL  TYPE 
Public 
Nonpublic 
Private 
Catholic 
BIA 

DODEA 

MODAL  AGE 
Younger 
At  Modal  Age 
Older 


68.8 

69.6 

69.9 

14.6 

14.1 

14.3 

12.2 

11.8 

11.2 

2.6 

2.6 

3.8 

1.8 

1.7 

0.7 

0.1 

0.2 

0.1 

21.8 

22.1 

21.6 

22.4 

21.8 

21.2 

25.7 

23.8 

24.2 

30.0 

32.3 

33.1 

4.2 

6.2 

6.6 

13.6 

19.5 

18.2 

7.4 

19.5 

25.2 

39.7 

44.0 

45.8 

33.2  . 

9.1 

2.5 

36.9 

26.6 

35.0 

38.1 

45.1 

36.3 

25.0 

28.3 

28.7 

87.5  • 

88.8 

87.7 

12.5 

11.2 

12.3 

4.8 

4.7 

4.4 

7.7 

6.6 

7.9 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.7 

0.6 

1.1 

59.3 

54.5 

63.6 

40.0 

44.9 

35.3 

0.0 

0.0 

49.2 

0.0 

0.0 

50.8 

0.0 

0.0 

74.1 

0.0 

0.0 

9.1 

0.0 

0.0 

7.0 

0.0 

0.0 

9.1 

0.0 

0.0 

0.5 

0.0 

0.0 

0.2 

0.0 

0.0 

24.4 

0.0 

0.0 

21.2 

0.0 

0.0 

30.4 

0.0 

0.0 

24.0 

0.0 

0.0 

2.7 

0.0 

0.0 

11.2 

0.0 

0.0 

20.8 

0.0 

0.0 

62.8 

0.0 

0.0 

0.0 

0.0 

0.0 

31.2 

0.0 

0.0 

41.3 

0.0 

0.0 

27.4 

0.0 

0.0 

83.0 

0.0 

0.0 

17.0 

0.0 

0.0 

5.4 

0.0 

0.0 

11.7 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

1.4 

0.0 

0.0 

70.9 

0.0 

0.0 

27.7 

‘Advanced  students  not  sampled  for  Grade  4 or  Grade  8. 
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Table  19-28 

Weighted  Percentage  of  Excluded  Students  in  the  Mathematics  Main  and  Estimation 
Samples  by  Subgroup  Classification,  Grades  4,  8,  and  12 


main  estimation 

Grade  4 Grade  8 Grade  12  Grade  4 Grade  8 Grade  12 

229,564  162,944  88,046 

62.0  62.1  67.9 

38.0  37.9  32.1 


TOTAL 

GENDER 

Male 

Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Asian  American 
American  Indian 
Unclassified 

REGION 

Northeast 

Southeast 

Central 

West 

PARENT’S  EDUCATION 
Less  Than  High  School 
High  School 

Greater  Than  High  School 
Graduated  College 
Unknown 

TYPE  OF  LOCATION 
Central  City 

Urban  Fringe/Large  Town 
Rural/Small  Town 

SCHOOL  TYPE 
Public 
Non  Public 
Private 
Catholic 
BIA 

DODEA 

MODAL  AGE 
Younger 
At  Modal  Age 
Older 


56.1 

65.5 

65.2 

16.4 

18.3 

15.9 

22.9 

11.2 

17.6 

2.6 

2.8 

1.0 

1.6 

1.4 

0.3 

0.3 

0.8 

0.0 

12.2 

22.0 

29.8 

25.0 

20.5 

19.7 

16.4 

24.6 

14.4 

46.4 

32.9 

36.2 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

28.8 

38.1 

27.9 

35.6 

35.6 

51.4 

35.6 

26.3 

20.8 

97.8 

99.4 

99.5 

2.2 

0.6 

0.5 

1.0 

0.0 

0.0 

1.1 

0.6 

0.5 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

2.4 

0.8 

41.0 

23.1 

36.2 

59.0 

74.5 

63.0 

180,417 

142,276 

110,197 

77.2 

56.8 

56.3 

22.8 

43.2 

43.7 

69.4 

64.3 

42.7 

11.8 

12.6 

24.4 

14.2 

19.2 

29.3 

2.5 

3.9 

3.6 

2.1 

0.0 

0.0 

0.0 

0.0 

0.0 

12.4 

22.1 

16.4 

19.0 

20.8 

13.3 

35.8 

18.1 

14.9 

32.8 

39.0 

55.4 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

23.5 

49.7 

51.1 

46.9 

24.5 

18.3 

29.5 

25.7 

30.6 

100.0 

100.0 

100.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

48.8 

33.8 

20.3 

51.2 

66.2 

79.7 

O 
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Table  19-29 

Weighted  Percentage  of  Excluded  Students  in  the  Mathematics  Theme  and  Advanced 
Samples  by  Subgroup  Classification,  Grades  4,  8,  and  12 


THEME  ADVANCED 

Grade  4 Grade  8 Grade  12  Grade  4^  Grade  8 Grade  12 

0 1,042  1,701 


TOTAL 

GENDER 

Male 

Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Asian  American 
American  Indian 
Unclassified 

REGION 
• Northeast 
Southeast 
Central 
West 

PARENT’S  EDUCATION 
Less  Than  High  School 
High  School 

Greater  Than  High  School 
Graduated  College 
Unknown 

TYPE  OF  LOCATION 
Central  City 

Urban  Fringe/Large  Town 
Rural/Small  Town 

SCHOOL  TYPE 
Public 
Nonpublic 
Private 
Catholic 
BIA 

DODEA 

MODAL  AGE 
Younger 
At  Modal  Age 
Older 


238,555 

160,860 

115,182 

58.3 

55.1 

57.1 

41.7 

44.9 

42.9 

42.0 

46.0 

67.7 

18.2 

25.3 

13.8 

31.3 

24.3 

14.8 

4.9 

1.7 

3.1 

2.8 

2.8 

0.6 

0.7 

0.0 

0.0 

12.7 

12.9 

28.5 

18.9 

31.0 

21.0 

25.7 

21.1 

16.3 

42.6 

35.0 

34.1 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

54.8 

31.3 

42.3 

36.5 

40.2 

40.1 

8.7 

28.5 

17.6 

98.6 

99.2 

95.2 

1.4 

0.8 

4.8 

0.3 

0.0 

4.8 

1.1 

0.8 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

2.7 

1.5 

0.4 

46.8 

30.7 

22.7 

50.6 

67.8 

76.8 

0.0 

100.0 

89.2 

0.0 

0.0 

10.8 

0.0 

0.0 

36.1 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0  . 

100.0 

63.9 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

41.3 

0.0 

0.0 

0.0 

0.0 

0.0 

26.5 

0.0 

100.0 

32.3 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

100.0 

41.0 

0.0 

0.0 

22.9 

0.0 

0.0 

36.1 

0.0 

100.0 

100.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

100.0 

59.0 

0.0 

0.0 

41.0 

' Advanced  students  not  sampled  for  Grade  4. 
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Figure  19-30 

Weighted  Percentage  of  Excluded  Students  in  the  Science  Main  and  Advanced  Samples  by  Subgroup 

Classification,  Grades  4,  8,  and  12 


Grade  4 

MAIN 
Grade  8 

Grade  12 

Grade  4 

ADVANCED 
Grade  8 

Grade  12 

TOTAL 

322,613 

164,891 

119,759 

0 

0 

n 

SEX 

Male 

63.2 

63.6 

61.9 

0.0 

0.0 

0.0 

Female 

36.4 

38.1 

38.1 

0.0 

0.0 

0.0 

RACE/ETHNICITY 

White 

49.1 

54.6 

54.1 

0.0 

0.0 

0.0 

Black 

16.6 

19.4 

22.4 

0.0 

0.0 

0.0 

Hispanic 

27.6 

21.6 

18.1 

0.0 

0.0 

0.0 

Asian  American 

5.3 

2.2 

4.2 

0.0 

0.0 

0.0 

American  Indian 

0.8 

2.0 

0.5 

0.0 

0.0 

0.0 

Unclassified 

0.5 

0.2 

0.7 

0.0 

0.0 

0.0 

REGION 

Northeast 

14.6 

12.7 

21.8 

0.0 

0.0 

0.0 

Southeast 

23.6 

23.3 

22.6 

0.0 

0.0 

0.0 

Central 

22.2 

21.9 

18.3 

0.0 

0.0 

0.0 

West 

39.6 

42.2 

37.2 

0.0 

0.0 

0.0 

PARENTS  EDUCATION 
Less  Than  High  School 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

High  School 

0.0 

0.5 

0.0 

0.0 

0.0 

0.0 

Greater  Than  High  School 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Graduated  College 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Unknown 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

TYPE  OF  LOCATION 

Central  City 

52.0 

29.1 

40.1 

0.0 

0.0 

0.0 

Urban  Fringe/Large  Town 

24.8 

33.6 

32.8 

0.0 

0.0 

0.0 

Rural/Small  Town 

23.3 

37.3 

27.1 

0.0 

0.0 

0.0 

SCHOOL  TYPE 
Public 

99.5 

99.4 

99.0 

0.0 

0.0 

0.0 

Nonpublic 

0.5 

0.6 

1.0 

0.0 

0.0 

0.0 

Private 

0.2 

0.0 

0.8 

0.0 

0.0 

0.0 

Catholic 

0.3 

0.6 

0.2 

0.0 

0.0 

0.0 

BIA 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

DoDEA 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

MODAL  AGE 

<Modal  Age 

0.9 

1.0 

1.1 

0.0 

0.0 

0.0 

=Modal  Age 

37.7 

25.7 

25.2 

0.0 

0.0 

0.0 

>Modal  Age 

61.3 

73.2 

73.7 

0.0 

0.0 

0.0 

ERIC 
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Table  19-31 

Weighted  Percentage  of  Students  in  the  Reading  and  Writing  Long-Term  Trend 
Sample  by  Type  of  Eligibility  and  Subgroup  Classification,  Age  9/Grade  4 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,170,010 

3,579,694 

2,119,331 

4,630,373 

GENDER 

Male 

49.5 

49.3 

46.3 

50.8 

Female 

50.5 

50.7 

53.7 

49.2 

RACE/ETHNICITY 

White 

66.7 

68.3 

67.5 

67.6 

Black 

15.4 

14.6 

15.1 

15.0 

Hispanic 

12.9 

12.7 

12.4 

13.0 

Asian  American 

2.6 

2.6 

3.5 

2.3 

American  Indian 

2.1 

1.5 

.1.3 

2.0 

Unclassified 

0.2 

0.2 

0.3 

0.2 

REGION 

Northeast 

23.5 

22.1 

26.0 

21.3 

Southeast 

23.9 

23.8 

22.2 

24.6 

Central 

24.6 

25.5 

22.1 

26.4 

West 

28.0 

28.6 

29.7 

27.7 

PARENT’S  EDUCATION 

Less  Than  High  School 

4.0 

4.4 

3.9 

4.4 

High  School 

15.3 

16.0 

14.5 

16.2 

Greater  Than  High  School 

5.0 

4.7 

4.6 

4.9 

Graduated  College 

39.5 

41.6 

42.1 

39.9 

Unknown 

35.5 

32.2 

34.1 

33.6 

TYPE  OF  LOCATION 

Central  City 

40.8 

42.5 

41.6 

41.8 

Urban  Fringe/Large  Town 

34.3 

33.4 

36.6 

32.6 

Rural/Small  Town 

24.8 

24.0 

21.8 

25.6 

SCHOOL  TYPE 

Public 

86.2 

86.3 

85.2 

86.7 

Nonpublic 

13.8 

13.7 

14.8 

13.3 

Private 

6.8 

6.2 

6.7 

6.4 

Catholic 

7.1 

7.4 

8.1 

6.9 

BIA 

0.0 

0.0 

0.0 

0.0 

DODEA 

0.0 

0.0 

0.0 

0.0 

411 


Table  19-32 

Weighted  Percentage  of  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13/Grade  8 


Age 


Grade Age  and  Grade Age  or  Grade 


TOTAL 

GENDER 

Male 

Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Asian  American 
American  Indian 
Unclassified 

REGION 

Northeast 

Southeast 

Central 

West 

PARENT’S  EDUCATION 
Less  Than  High  School 
High  School 

Greater  Than  High  School 
Graduated  College 
Unknown 

TYPE  OF  LOCATION 
Central  City 

Urban  Fringe/Large  Town 
Rural/Small  Town 

SCHOOL  TYPE 
Public 
Nonpublic 
Private 
Catholic 
BIA 

DODEA 


3,173,938 

3,465,078 

48.6 

50.3 

51.4 

49.7 

66.6 

69.1 

15.1 

14.4 

12.7 

12.0 

3.5 

2.8 

2.0 

1.7 

0.1 

0.1 

23.1 

22.3 

25.5 

25.0 

21.8 

23.4 

29.6 

29.3 

5.3 

7.0 

28.8 

28.5 

10.4 

10.6 

45.0 

45.0 

10.1 

8.5 

36.2 

36.0 

35.7 

33.9 

28.1 

30.1 

88.5 

89.4 

11.2 

9.9 

4.8 

4.6 

6.3 

5.2 

0.4 

0.7 

0.0 

0.0 

1,943,322 

4,695,694 

44.4 

51.6 

55.6 

48.4 

68.9 

67.4 

13.6 

15.2 

12.2 

12.4 

3.8 

2.9 

1.5 

2.0 

0.1 

0.1 

24.7 

21.8 

23.8 

25.9 

21.1 

23.3 

30.4 

29.0 

5.3 

6.5 

27.8 

28.9 

11.0 

10.3 

47.3 

44.1 

8.4 

9.7 

35.4 

36.4 

38.2 

33.3 

26.4 

30.3 

87.9 

89.4 

11.7 

10.0 

5.7 

4.3 

6.0 

5.7 

0.4 

0.6 

0.0 

0.0 

O 

ERIC 
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Table  19-33 

Weighted  Percentage  of  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  17/Grade  11 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

total 

3,224,505 

3,160,512 

1,977,398 

4,407,619 

GENDER 

Male 

51.4 

52.4 

48.9 

53.3 

Female 

48.6 

47.6 

51.1 

46.7 

RACE/ETHNICITY 

White 

69.3 

68.3 

74.2 

66.4 

Black 

14.4 

15.1 

11.7 

16.1 

Hispanic 

11.7 

12.4 

9.9 

13.0 

Asian  American 

3.1 

2.9 

2.6 

3.2 

American  Indian 

1.4 

1.2 

1.3 

1.3 

Unclassified 

0.1 

0.2 

0.2 

0.1 

REGION 

Northeast 

22.3 

24.9 

23.9 

23.5 

Southeast 

24.2 

22.7 

21.4 

24.4 

Central 

24.5 

24.5 

26.0 

23.8 

West 

29.0 

27.9 

28.8, 

28.3 

PARENT’S  EDUCATION 
Less  Than  High  School 

6.8 

6.7 

4.9 

7.6 

High  School 

26.7 

26.0 

24.7 

27.1 

Greater  Than  High  School 

18.0 

18.7 

19.8 

17.7 

Graduated  College 

44.7 

45.2 

47.9 

43.6 

Unknown 

2.8 

2.7 

2.1 

3.0 

TYPE  OF  LOCATION 

Central  City 

34.7 

34.2 

32.3 

35.4 

Urban  Fringe/Large  Town 

39.2 

39.5 

41.4 

38.5 

Rural/Small  Town 

26.1 

26.3 

26.4 

26.1 

SCHOOL  TYPE 

Public 

91.7 

90.8 

90.7 

91.5 

Nonpublic 

8.1 

9.0 

9.0 

8.3 

Private 

3.5 

3.4 

3.6 

3.4 

Catholic 

4.6 

5.6 

5.4 

4.9 

BIA 

0.2 

0.2 

0.4 

0.2 

DODEA 

0.0 

0.0 

0.0 

0.0 
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Table  19-34 

Weighted  Percentage  of  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  9‘ 


Age Grade  Age  and  Grade  Age  or  Grade 


TOTAL 

3,320,984 

2,207,888 

2,207,888 

3,320,984 

GENDER 

Male 

49.6 

48.0 

48.0 

49.6 

Female 

50.4 

52.0 

52.0 

50.4 

RACE/ETHNICITY 

White 

69.0 

68.1 

68.1 

69.0 

Black 

14;3 

15.5 

15.5 

14.3 

Hispanic 

12.4 

11.6 

11.6 

12.4 

Asian  American 

2.3 

2.9 

2.9 

2.3 

American  Indian 

1.9 

1.8 

1.8 

1.9 

Unclassified 

0.2 

0.2 

0.2 

0.2 

REGION 

Northeast 

21.7 

25.7 

25.7 

21.7 

Southeast 

23.4 

22.5 

22.5 

23.4 

Central 

25.0 

21.4 

21.4 

25.0 

West 

30.0 

30.4 

30.4 

30.0 

PARENT’S  EDUCATION 

Less  Than  High  School 

3.9 

3.8 

3.8 

3.9 

High  School 

12.5 

12.2 

12.2 

12.5 

Greater  Than  High  School 

7.1 

7.9 

7.9 

7.1 

Graduated  College 

42.7 

45.0 

■ 45.0 

42.7 

Unknown 

33.1 

30.6 

30.6 

33.1 

Type  Of  Location 

Central  City 

42.3 

43.0 

43.0 

42.3 

Urban  Fringe/Large  Town 

32.4 

34.3 

34.3 

32.4 

Rural/Small  Town 

25.4 

22.7 

22.7 

25.4 

SCHOOL  TYPE 

Public 

86.8 

86.3 

86.3 

86.8 

Nonpublic 

12.9 

13.4 

13.4 

12.9 

Private 

4.0 

3.8 

3.8 

4.0 

Catholic 

8.9 

9.5 

9.5 

8.9 

BIA 

0.3 

0.3 

0.3 

0.3 

DODEA 

0.0 

0.0 

0.0 

0.0 

' Note:  Since  this  is  an  age-only  sample,  the  number  of  students  who  are  age-eligible  only  will  be  the  same  as  the  number  of 
students  who  are  age-  or  grade-eligible.  Likewise,  the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the 
number  of  students  who  are  both  age-  and  grade-eligible. 
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Table  19-35 

Weighted  Percentage  of  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13^ 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,360,572 

2,128,872 

2,128,872 

3,360,572 

GENDER 

Male 

48.5 

45.2 

45.2 

48.5 

Female 

51.5 

54.8 

54.8 

51.5 

RACE/ETHNICITY 

White 

68.6 

68.0 

68.0 

68.6 

Black 

14.3 

14.7 

14.7 

14.3 

Hispanic 

12.2 

11.3 

11.3 

12.2 

Asian  American 

3.6 

4.4 

4.4 

3.6 

American  Indian 

1.3 

1.4 

1.4 

1.3 

Unclassified 

0.1 

0.1 

0.1 

0.1 

REGION 

Northeast 

22.0 

25.0 

25.0 

22.0 

Southeast 

24.9 

22.9 

22.9 

24.9 

Central 

23.1 

21.9 

21.9 

23.1 

West 

30.0 

30.2 

30.2 

30.0 

PARENT’S  EDUCATION 

Less  Than  High  School 

5.5 

4.3 

4.3 

5.5 

High  School 

22.7 

22.1 

22.1 

22.7 

Greater  Than  High  School 

16.7 

18.7 

18.7 

16.7 

Graduated  College 

45.0 

46.7 

46.7 

45.0 

Unknown 

9.7 

7.9 

7.9 

9.7 

TYPE  OE  LOCATION 

Central  City 

35.7 

36.2 

36.2 

35.7 

Urban  Fringe/Large  Town 

35.9 

38.1 

38.1 

35.9 

Rural/Small  Town 

28.3  ■ 

25.6 

25.6 

28.3 

SCHOOL  TYPE 

Public 

88.8 

88.2 

88.2 

88.8 

Nonpublic 

11.2 

11.8 

11.8 

11.2 

Private 

4.5 

5.1 

5.1 

4.5 

Catholic 

6.7 

6.7 

6.7 

6.7 

BIA 

0.0 

0.0 

0.0 

0.0 

DODEA 

0.0 

0.0 

0.0 

0.0 

‘ Note:  Since  this  is  an  age-only  sample,  the  number  of  students  who  are  age-eligible  only  will  be  the  same  as  the  number  of 
students  who  are  age-  or  grade-eligible.  Likewise,  the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the 
number  of  students  who  are  both  age-  and  grade-eligible. 
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Table  19-36 

Weighted  Percentage  of  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  17' 


Age 

Grade 

Age  and  Grade 

Age  or  Grade 

TOTAL 

3,185,309 

2,250,256 

2,250,256 

3,185,309 

GENDER 

Male 

49.5 

47.0 

47.0 

49.5 

Female 

50.5 

53.0 

53.0 

50.5 

RACE/ETHNICITY 

White 

69.3 

73.8 

73.8 

69.3 

Black 

14.5 

12.6 

12.6 

14.5 

Hispanic 

11.6 

9.7 

9.7 

11.6 

Asian  American 

3.5 

3.0 

3.0 

3.5 

American  Indian 

0.8 

0.7 

0.7 

0.8 

Unclassified 

0.2 

. 0.2 

0.2 

0.2 

REGION 

Northeast 

23.4 

24.1 

24.1 

23.4 

Southeast 

22.4 

22.1 

22.1 

22.4 

Central 

24.5 

24.2 

24.2 

24.5 

West 

29.7 

29.6 

29.6 

29.7 

PARENT’S  EDUCATION 

Less  Than  High  School 

6.4 

4.4 

4.4 

6.4 

High  School 

20.9 

19.6 

19.6 

20.9 

Greater  Than  High  School 

23.8 

24.4 

24.4 

23.8 

Graduated  College 

46.0 

49.2 

49.2 

46.0 

Unknown 

2.0 

1.5 

1.5 

2.0 

TYPE  OF  LOCATION 

Central  City 

36.9 

35.3 

35.3 

36.9 

Urban  Fringe/Large  Town 

37.9 

39.4 

39.4 

37.9 

Rural/Small  Town 

25.1 

25.3 

25.3 

25.1 

SCHOOL  TYPE 

Public 

91.4 

90.7 

90.7 

91.4 

Nonpublic 

8.6 

9.3 

9.3 

8.6 

Private 

3.5 

3.7 

3.7 

3.5 

Catholic 

5.1 

5.6 

5.6 

5.1 

BIA 

0.0 

0.0 

0.0 

0.0 

DODEA 

0.0 

0.0 

0.0 

0.0 

' Note:  Since  this  is  an  age-only  sample,  the  number  of  students  who  are  age-eligible  only  will  be  the  same  as  the  number  of 
students  who  are  age-  or  grade-eligible.  Likewise,  the  number  of  students  who  are  grade-eligible  only  will  be  the  same  as  the 
number  of  students  who  are  both  age-  and  grade-eligible. 
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Table  19-37 

Weighted  Percentage  of  Excluded  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample  by  Type 
of  Eligibility  and  Subgroup  Classification,  Age  9/Grade  4 


Eligible  by 


Age  & 

Age  or 

Age 

Grade 

Grade 

Grade 

SEX 


Male 

61.2 

63.6 

57.7 

64.0 

Female 

38.8 

36.4 

42.3 

36.0 

RACE/ETHNICITY 

White 

48.3 

57.6 

42.7 

57.5 

Black 

18.3 

16.9 

14.9 

17.8 

Hispanic 

24.3 

19.0 

30.1 

18.5 

Asian  American 

7.6 

4.7 

10.4 

4.5 

American  Indian 

0.8 

1.6 

1.3 

1.4 

Unclassified 

0.7 

0.1 

0.6 

0.2 

REGION 

Northeast 

9.4 

13.1 

7.8 

12.9 

Southeast 

26.9 

36.4 

23.5 

35.8 

Central 

17.9 

18.5 

11.7 

19.7 

West 

45.7 

32.0 

57.0 

31.5 

TYPE  OF  LOCATION 

Central  City 

50.2 

42.8 

55.0 

42.8 

Urban  Fringe/Large  Town 

34.5 

28.7 

36.7 

29.1 

Rural/Small  Town 

15.3 

28.5 

8.3 

28.1 

SCHOOL  TYPE 

Public 

98.4 

99.7 

98.7 

99.4 

Nonpublic 

1.3 

0.3 

1.3 

0.4 

Private 

0.0 

0.0 

0.0 

0.0 

Catholic 

1.3 

0.3 

1.3 

0.4 

BIA 

0.0 

0.0 

0.0 

0.0 

DoDea 

0.0 

0.0 

0.0 

0.0 

ESTIMATED  TOTAL 

POPULATION 

106,503 

266,020 

64,398 

308,125 

ERIC 
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Table  19-38 

Weighted  Percentage  of  Excluded  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample  by  Type 
of  Eligibility  and  Subgroup  Classification,  Age  13/Grade  8 


Eligible  by 


Age 

Grade 

Age  & 
Grade 

Age  or 
Grade 

SEX 

Male 

68.4 

67.3 

77.6 

66.8 

Female 

31.6 

32.7 

22.4 

33.2 

RACE/ETHNICITY 

White 

57.5 

69.5 

60.5 

66.6 

Black 

18.4 

13.5 

9.6 

15.4 

Hispanic 

17.8 

13.1 

23.4 

13.7 

Asian  American 

3.4 

1.2 

3.7 

1.6 

American  Indian 

2.4 

2.6 

2.8 

2.5 

Unclassified 

0.4 

0.0 

0.0 

0.1 

REGION 

Northeast 

20.3 

17.4 

21.8 

17.9 

Southeast 

34.1 

32.9 

23.1 

34.1 

Central 

15.1 

28.6 

15.4 

25.6 

West 

30.4 

21.1 

39.7 

22.4 

TYPE  OF  LOCATION 

Central  City 

45.4 

40.0 

41.3 

41.5 

Urban  Fringe/Large  Town 

26.7 

21.2 

39.4 

21.4 

Rural/Small  Town 

27.9 

38.8 

19.4 

37.1 

SCHOOL  TYPE 

Public 

96.8 

96.9 

97.4 

96.8 

Nonpublic 

1.3 

0.5 

0.0 

0.8 

Private 

0.9 

0.5 

0.0 

0.7 

Catholic 

0.4 

0.0 

0.0 

0.1 

BIA 

1.9 

2.6 

2.6 

2.4 

DoDea 

0.0 

0.0 

0.0 

0.0 

ESTIMATED  TOTAL 
POPULATION 

87,533 

223,797 

24,049 

287,281 

<5  118 

ERIC 
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Table  19-39 

Weighted  Percentage  of  Excluded  Students  in  the  Reading  and  Writing  Long-Term  Trend  Sample  by  Type 
of  Eligibility  and  Subgroup  Classification,  Age  17/Grade  11 


Eligible  by 


Age  & 

Age  or 

Age 

Grade 

Grade 

Grade 

SEX 


Male 

64.4 

66.7 

47.6 

67.9 

Female 

35.6 

33.3 

52.4 

32.1 

RACE/ETHNICITY 

White 

61. 1 

67.6 

71.3 

64.1 

Black 

23.1 

14.0 

19.4 

17.6 

Hispanic 

II.7 

12.3 

6.4 

12.7 

Asian  American 

2.7 

3.2 

0.6 

3.3 

American  Indian 

0.9 

2.9 

2.3 

2.0 

Unclassified 

0.5 

0.0 

0.0 

0.2 

REGION 

Northeast 

15.0 

25.0 

23.7 

20.5 

Southeast 

36.5 

23.1 

26.4 

29.0 

Central 

19.9 

19.5 

I8.I 

19.9 

West 

28.6 

32.3 

31.8 

30.6 

TYPE  OF  LOCATION 

Central  City 

29.6 

30.9 

26.3 

30.8 

Urban  Fringe/Large  Town 

38.3 

46.3 

52.2 

41.8 

Rural/Small  Town 

32.2 

22.9 

21.6 

27.4 

SCHOOL  TYPE 

Public 

100.0 

97.8 

100.0 

98.6 

Nonpublic 

0.0 

0.6 

0.0 

0.4 

Private 

0.0 

0.0 

0.0 

0.0 

Catholic 

0.0 

0.6 

0.0 

0.4 

BIA 

0.0 

1.7 

0.0 

1. 1 

DoDea 

0.0 

0.0 

0.0 

0.0 

ESTIMATED  TOTAL 

POPULATION 

121,771 

167,734 

29,838 

259,667 

ERIC 


452 
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Table  19-40 

Weighted  Percentage  of  Excluded  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample  by 

Type  of  Eligibility  and  Subgroup  Classification,  Age  9 


Age 

uy 

Age  & 
Grade  Grade 

Age  or 
Grade 

SEX 

Male 

62.0 

61.0 

61.0 

62.0 

Female 

38.0 

39.0 

39.0 

38.0 

RACE/ETHNICITY 

White 

47.5 

46.4 

46.4 

47.5 

Black 

19.5 

13.2 

13.2 

19.5 

Hispanic 

25.4 

30.7 

30.7 

25.4 

Asian  American 

6.5 

8.8 

8.8 

6.5 

American  Indian 

0.3 

0.3 

0.3 

0.3 

Unclassified 

0.8 

0.6 

0.6 

0.8 

REGION 

Northeast 

12.0 

11.7 

11.7 

12.0 

Southeast 

33.2 

22.6 

22.6 

33.2 

Central 

18.0 

15.6 

15.6 

18.0 

West 

36.8 

50.0 

50.0 

36.8 

TYPE  OF  LOCATION 

Central  City 

54.8 

61.0 

61.0 

54.8 

Urban  Fringe/Large  Town 

27.2 

30.0 

30.0 

27.2 

Rural/Small  Town 

18.0 

9.0 

9.0 

18.0 

SCHOOL  TYPE 

Public 

98.5 

99.2 

99.2 

98.5 

Nonpublic 

1.5 

0.8 

0.8 

1.5 

Private 

0.0 

0.0 

0.0 

0.0 

Catholic 

1.5 

0.8 

0.8 

1.5 

BIA 

0.0 

0.0 

0.0 

0.0 

DoDea 

0.0 

0.0 

0.0 

0.0 

ESTIMATED  TOTAL 

POPULATION 

173,491 

91,294 

91,294 

173,491 

O 20 

ERIC 
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Table  19-41 

Weighted  Percentage  of  Excluded  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample  by 

Type  of  Eligibility  and  Subgroup  Classification,  Age  13 


Eligible  by 


Age 

Grade 

Age  & 
Grade 

Age  or 
Grade 

SEX 

Male 

64.6 

59.0 

59.0 

64.6 

Female 

35.4 

41.0 

41.0 

35.4 

RACE/ETHNICITY 

White 

54.4 

55.9 

55.9 

54.4 

Black 

20.1 

14.1 

14.1 

20.1 

Hispanic 

21.2 

26.9 

26.9 

21.2 

Asian  American 

3.4 

2.8 

2.8 

3.4 

American  Indian 

0.3 

0.3 

0.3 

0.3 

Unclassified 

0.5 

0.0 

0.0 

0.5 

REGION 

Northeast 

19.9 

23.1 

23.1 

19.9 

Southeast 

31.5 

24.1 

24.1 

31.5 

Central 

21.0 

19.0 

19.0 

21.0 

West 

27.6 

33.8 

33.8 

27.6 

TYPE  OF  LOCATION 

Central  City 

44.2 

36.8 

36.8 

44.2 

Urban  Fringe/Large  Town 

25.9 

38.3 

38.3 

25.9 

Rural/Small  Town 

30.0 

24.9 

24.9 

30.0 

SCHOOL  TYPE 

Public 

99.6 

100.0 

100.0 

99.6 

Nonpublic 

0.4 

0.0 

0.0 

0.4 

Private 

0.4 

0.0 

0.0 

0.4 

Catholic 

0.0 

0.0 

0.0 

0.0 

BIA 

0.0 

0.0 

0.0 

0.0 

DoDea 

0.0 

0.0 

0.0 

0.0 

ESTIMATED  TOTAL 

POPULATION 

146,608 

43,419 

43,419 

146,608 
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Table  19-42 

Weighted  Percentage  of  Excluded  Students  in  the  Mathematics  and  Science  Long-Term  Trend  Sample  by 

Type  of  Eligibility  and  Subgroup  Classification,  Age  17 


Eligible  by 


Age 

Grade 

Age  & 
Grade 

Age  or 
Grade 

SEX 

Male 

67.4 

62.0 

62.0 

67.4 

Female 

32.6 

38.0 

38.0 

32.6 

RACE/ETHNICITY 

White 

61.0 

71.0 

71.0 

61.0 

Black 

18.6 

16.1 

16.1 

18.6 

Hispanic 

15.4 

10.8 

10.8 

15.4 

Asian  American 

4.0 

1.7 

1.7 

4.0 

American  Indian 

1.0 

0.4 

0.4 

1.0 

Unclassified 

0.0 

0.0 

0.0 

0.0 

REGION 

Northeast 

22.1 

36.4 

36.4 

22.1 

Southeast 

30.5 

16.8 

16.8 

30.5 

Central 

17.9 

17.6 

17.6 

17.9 

West 

29.6 

29.2 

29.2 

29.6 

TYPE  OF  LOCATION 

Central  City 

37.3 

33.4 

33.4 

37.3 

Urban  Fringe/Large  Town 

36.3 

47.7 

47.7 

36.3 

Rural/Small  Town 

26.4 

18.9 

18.9 

26.4 

SCHOOL  TYPE 

Public 

98.6 

100.0 

100.0 

98.6 

Nonpublic 

1.4 

0.0 

0.0 

1.4 

Private 

1.4 

0.0 

0.0 

1.4 

Catholic 

0.0 

0.0 

0.0 

0.0 

BIA 

0.0 

0.0 

0.0 

0.0 

DoDea 

0.0 

0.0 

0.0 

0.0 

ESTIMATED  TOTAL 

POPULATION 

131,897 

38,085 

38,085 

131,897 

O 422 
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Appendix  A 


PARTICIPANTS  IN  THE  OBJECTIVES  AND  ITEM 
DEVELOPMENT  PROCESS 


The  National  Assessment  of  Educational  Progress  extends  its  deep  appreciation  to  all  those 
individuals  who  participated  in  the  development  of  the  framework,  objectives,  and  items  for  the  1996 
national  assessment. 


Frank  Betts 

Ed  Esty 
James  Gates 

Alice  Gill 

Joyce  McCray 
Henry  Pollack 

John  Porter 
Rodney  Riffel 
Ramsay  Selden 

Charles  Watson 

Michael  Webb 
Gene  Wilhoit 


Mathematics  Project  Steering  Committee 

Director,  Curriculum  Technology  Center,  Association  for  Supervision  and 
Curriculum  Development,  Alexandria,  Virginia 

Senior  Staff  Officer,  Mathematical  Sciences  Education  Board,  Washington,  DC 

Executive  Director,  National  Council  of  Teachers  of  Mathematics,  Reston, 
Virginia 

Coordinator,  Thinking  Mathematics  Dissemination,  American  Federation  of 
Teachers,  Cleveland,  Ohio 

- t 

Executive  Director,  Council  for  American  Private  Education,  Washington,  DC 

Visiting  Professor,  Teachers  College,  Columbia  (formerly  with  Bell  Labs),  New 
York,  New  York 

Director,  Education  and  University  Relations,  IBM,  Armonk,  New  York 

Senior  Professional  Associate,  National  Education  Association,  Washington,  DC 

Director,  State  Education  Assessment  Center,  Council  of  Chief  State  School 
Officers,  Washington,  DC 

President,  Association  of  State  Supervisors  of  Mathematics,  Arkansas  Director 
of  Mathematics,  Little  Rock,  Arkansas 

National  Education  Director,  National  Urban  League,  New  York,  New  York 

Executive  Director,  National  Association  of  State  Boards  of  Education, 
Alexandria,  Virginia 
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Science  Project  Steering  Committee 


William  O.  Baker 
Mary  Louise  Bellamy 
Frank  Betts 

William  B.  Campbell 

Glenn  A.  Crosby 

Gerald  Difford 
Janice  Earle 

John  Fowler 

Johnnie  Hamilton 
Elam  Hertzler 
Ann  Kahn 
Douglas  Lapp 

John  Layman 

Harold  Pratt 

Judith  Tomey-Purta 

Douglas  Reynolds 

Bella  Rosenberg 
Jane  Sisk 


Retired,  AT&T  Bell  Laboratories,  Murray  Hill,  New  Jersey 

Education  Director,  National  Association  of  Biology  Teachers,  Reston,  Virginia 

Director,  Curriculum  Technology  Center,  Association  for  Supervision  and 
Curriculum  Development,  Alexandria,  Virginia 

Executive  Director,  National  Industry  Council  for  Science  Education,  College 
Park,  Maryland 

Professor,  Chemistry  Department,  Washington  State  University,  Pullman, 
Washington 

Executive  Director,  Colorado  Association  of  Schools,  Englewood,  Colorado 

Director,  Center  on  Educational  Equity,  National  Association  of  State  Boards  of 
Education,  Alexandria,  Virginia 

Director,  Triangle  Coalition  for  Science  and  Technology,  College  Park, 

Maryland 

Principal,  Franklin  Intermediate  School,  Chantilly,  Virginia 

Retired,  Secretary’s  Commission  on  Achieving  Necessary  Skills,  Washington,  DC 

Past  President,  Parent  Teachers  Association,  Fairfax,  Virginia 

Director,  National  Science  Resources  Center,  Smithsonian  Institution, 
Washington,  DC 

Director,  Science  Teaching  Center,  University  of  Maryland,  College  Park, 
Maryland 

Executive  Director,  Science  and  Technology  Management,  Jefferson  County 
Schools,  Golden,  Colorado 

Department  of  Human  Development,  University  of  Maryland,  College  Park, 
Maryland 

Chief,  Bureau  of  Science  Education,  State  Department  of  Education,  Albany, 

New  York 

Assistant  to  the  President,  American  Federation  of  Teachers,  Washington,  DC 
Biology  Teacher,  Calloway  County  High  School,  Murray,  Kentucky 
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Mathematics  Project  Planning  Committee 


Martha  Baca 

Glen  Blume 
Dave  Heckman 

Audrey  Jackson 
Pat  Kenney 

Donna  Long 
Larry  Wiley 


Supervisor,  K-12  Public  Schools,  Roosevelt  School  District  #66,  Phoenix, 
Arizona 

Associate  Professor,  Penn  State  LFniversity,  LFniversity  Park,  Pennsylvania 

Private  School  Teacher  and  Presidential  Award  Recipient,  1987,  Monmouth, 
Maine 

Middle  School  Mathematics  Supervisor  and  Teacher,  St.  Louis,  Missouri 

Researcher,  Learning  Research  and  Development  Center,  University  of 
Pittsburgh,  Pennsylvania 

Indiana  State  Supervisor  of  Mathematics,  Indianapolis,  Indiana 
New  Jersey  State  Supervisor  of  Mathematics,  Trenton,  New  Jersey 
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Science  Project  Planning  Committee 


Andrew  Ahlgren 

Bill  Aldridge 
J.  Myron  Atkin 
Joan  Boykoff  Baron 

Audrey  Champagne 

Sally  Crissman 

Edmund  W.  Gordon 
Henry  Heikkinen 
George  Hein 
Joseph  L.  Premo 
Senta  A.  Raizen 
James  Robinson 
Thomas  P.  Sachse 

Gary  E.  Skaggs 
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Associate  Director,  Project  2061,  American  Association  for  the  Advancement  of 
Science,  Washington,  DC 

Executive  Director,  National  Science  Teachers’  Association,  Washington,  DC 

Professor,  School  of  Education,  Stanford  University,  Stanford,  California 

Assessment  Coordinator,  Connecticut  Common  Core  of  Learning,  Bureau  of 
Evaluation  and  Student  Assessment,  Connecticut  Department  of  Education, 
Hartford,  Connecticut 

Professor,  School  of  Education,  State  University  of  New  York,  Albany,  New 
York 

Lowers  School  Head/Science  Teacher,  Shady  Hill  School,  Cambridge, 
Massachusetts 

Professor,  Department  of  Psychology,  Yale  University,  New  Haven,  Connecticut 

Director,  M.A.S.T.  Center,  University  of  Northern  Colorado,  Greeley,  Colorado 

Professor,  Lesley  College,  Cambridge,  Massachusetts 

Science  Consultant,  Minneapolis  Public  Schools,  Minneapolis,  Minnesota 

Director,  National  Center  for  Improving  Science  Education,  Washington,  DC 

Retired,  Curriculum  and  Evaluation,  Boulder  Valley  Schools,  Boulder,  Colorado 

Manager,  Math  and  Science  Unit,  California  Department  of  Education, 
Sacramento,  California 

Test  Development  Analyst,  Office  of  Research  and  Evaluation,  Fairfax  County 
Public  Schools,  Falls  Church,  Virginia 
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Mathematics  Project  Staff 


The  College  Board 
Robert  Orrill 
Valarie  French 

Bruce  Kennedy 


Executive  Director,  Office  of  Academic  Affairs 

Director  of  Curriculum  and  Instructional  Development,  Office  of  Academic 
Affairs 

Office  Manager,  Office  of  Academic  Affairs 


Consultants  to  the  College  Board 

John  Dossey  Project  Director,  Distinguished  University  Professor  of  Mathematics,  Illinois 

State  University,  Normal,  Illinois 

Cathy  Seeley  Project  Coordinator,  Mathematics  Consultant,  Former  Texas  Director  of 

Mathematics,  Austin,  Texas 
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Science  Proiect  Staff 

Ramsay  W.  Selden 

Director,  State  Education  Assessment  Center,  Council  of  Chief  State  School 
Officers 

Richard  C.  Clark 

Consensus  Coordinator,  Council  of  Chief  State  School  Officers 

Senta  A.  Raizen 

Director,  National  Center  for  Improving  Science  Education 

Julia  H.  Mitchell 

Associate  Research  Scientist,  American  Institutes  for  Research 

Bonnie  L.  Verrico 

Administrative  Assistant,  Council  of  Chief  State  School  Officers 

Science  Achievement  Levels  Panel 


Andrew  Ahlgren 

Audrey  Champagne 

Richard  C.  Clark 

Yvonne  Curbeam 
Joseph  Premo 
Thomas  Preston 
Senta  Raizen 
Dwight  Sieggreen 
William  Spooner 

Douglas  Wagner 
Sylvia  Ware 


Associate  Director,  Project  2061,  American  Association  for  the  Advancement  of 
Science,  Washington,  DC 

Professor,  School  of  Education,  State  University  of  New  York,  Albany,  New 
York 

Consensus  Coordinator  and  Science  Coordinator,  Mirinesota  Department  of 
Education,  St.  Paul,  Minnesota 

Science  Department  Chair,  Dunbar  High  School,  Baltimore,  Maryland 
Science  Consultant,  New  Hope,  Minnesota 

Science  Teacher,  Frick  International  Studies  Academy,  Pittsburgh,  Pennsylvania 

Director,  National  Center  for  Improving  Science  Education,  Washington,  DC 

Science  Teacher,  Cooke  Middle  School,  Northville,  Michigan 

Chief  Consultant,  Science  Education,  North  Carolina  Department  of  Public 
Instruction,  Raleigh,  North  Carolina 

Science  Teacher,  Emmanuel  Lutheran  School,  St.  Charles,  Missouri 
Director,  Education  Department,  American  Chemical  Society,  Washington,  DC 
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Appendix  B 


REPORTING  SUBGROUPS 
FOR  THE  1996  NAEP  ASSESSMENT 


Results  for  the  1996  assessment  were  reported  for  student  subgroups  defined  by  gender, 
race/ethnicity,  type  of  location,  parents’  level  of  education,  participation  in  the  National  School  Lunch 
Program,  eligibility  of  Title  I funding,  and  geographical  region.  The  following  explains  how  each  of 
these  subgroups  was  derived. 


DSEX  (Gender) 

The  variable  SEX  is  the  gender  of  the  student  being  assessed,  as  taken  from  school  records.  For  a 
few  students,  data  for  this  variable  was  missing  and  was  imputed  by  ETS  after  the  assessment.  The 
resulting  variable  DSEX  contains  a value  for  every  student  and  is  used  for  gender  comparisons  among 
students. 


DRACE  (Race/Ethnicity  for  Main  and  Long-Term  Trend  Assessments) 

The  variable  DRACE  is  an  imputed  definition  of  race/ethnicity,  derived  from  up  to  three  sources 
of  information.  This  variable  is  used  for  race/ethnicity  subgroup  comparisons  in  the  1996  main 
assessments  (science  and  mathematics),  in  the  1996  long-term  trend  samples  (reading,  mathematics,  and 
science),  and  in  the  1996  writing  trend  samples  (see  also  “observed  race/ethnicity”  below).  Two  items 
from  the  student  demographics  questionnaire  were  used  in  the  determination  of  derived  race/ethnicity: 


Demographic  Item  Number  2: 


2.  If  you  are  Hispanic,  what  is  your  Hispanic  background? 
<=)  lam  not  Hispanic. 

o Mexican,  Mexican  American,  or  Chicano 
<=)  Puerto  Rican 
<=)  Cuban 

<=>  Other  Spanish  or  Hispanic  background 
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Students  who  responded  to  Item  Number  2 by  filling  in  the  second,  third,  fourth,  or  fifth  oval 
were  considered  Hispanic.  For  students  who  filled  in  the  first  oval,  did  not  respond  to  the  item  or 
provided  information  that  was  illegible  or  could  not  be  classified,  responses  to  item  number  1 were 
examined  in  an  effort  to  determine  race/ethnicity.  Item  Number  1 read  as  follows: 


Demographic  Item  Number  1 : 

1.  Which  best  describes  you? 

<=>  White  (not  Hispanic) 
o Black  (not  Hispanic) 

<=>  Hispanic  ( Hispanic”  means  someone  who  is  Mexican,  Mexican 
American,  Chicano,  Puerto  Rican,  Cuban,  or  from  some  other 
Spanish  or  Hispanic  background.) 

<=>  Asian  or  Pacific  Islander  (“Asian  or  Pacific  Islander”  means 
someone  who  is  Chinese,  Japanese,  Korean,  Filipino,  Vietnamese, 
or  from  some  other  Asian  or  Pacific  Island  background.) 

<=>  American  Indian  or  Alaskan  Native  (“American  Indian  or  Alaskan 
Native”  means  someone  who  is  from  one  of  the  American  Indian 
tribes,  or  one  of  the  original  people  of  Alaska.) 
o Other  (What?) 

Students’  race/ethnicity  was  then  assigned  to  correspond  with  their  selection.  For  students  who 
filled  in  the  sixth  oval  (Other),  provided  illegible  information  or  information  that  could  not  be  classified, 
or  did  not  respond  at  all,  race/ethnicity  as  provided  from  school  records  was  used. 

Denved  race/ethnicity  could  not  be  determined  for  students  who  did  not  respond  to  background  items  1 
or  2 and  for  whom  race/ethnicity  was  not  provided  by  the  school. 


RACE  (Observed  Race/Ethnicity) 


The  variable  RACE  is  the  race/ethnicity  of  the  student  being  assessed  as  observed  and  recorded 
by  the  exercise  administrator.  Observed  race/ethnicity  was  used  in  NAEP  assessments  before  1984  (see 
also  “self-identified  race/ethnicity”  above).  Observed  race/ethnicity  is  used  for  all  race/ethnicity 
subgroup  trend  comparisons  for  which  the  starting  long-term  trend  point  comes  from  pre-1984 
assessment  data. 
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TOL8  (Type  of  Location) 

TOL5 

TOL3 


The  variable  TOL8  is  used  by  NAEP  to  provide  information  about  the  type  of  location  in  which 
schools  are  located.  The  variable  is  defined  using  population  size  information  from  the  1990  Census  and 
the  definitions  of  Metropolitan  Statistical  Areas  (MSAs)  as  of  June  1995.  There  are  eight  categories  for 
TOL8: 

1 Large  Central  City  a central  city  of  an  MSA  with  a population  greater  than  or  equal 

to  400,000,  or  a population  density  greater  than  or  equal  to  6,000 
persons  per  square  mile 

2 Midsize  Central  City  a central  city  of  an  MSA  but  not  designated  as  a large  city 


3 Urban  Fringe  of 
Large  City 


a place  within  an  MSA  of  a large  central  city  and  defined  as 
urban  by  the  U.S.  Bureau  of  Census 


4 Urban  Fringe  of  a 
Midsize  City 

5 Large  Town 

6 Small  Town 


a place  within  an  MSA  of  a midsize  central  city  and  defined 
as  urban  by  the  U.S.  Bureau  of  Census 

a place  not  within  an  MSA,  but  with  a population  greater  than  or 
equal  to  25,000  and  defined  as  urban  by  the  U.S.  Bureau  of  Census 

a place  not  within  an  MSA,  with  a population  less  than  25,000,  but 
greater  than  or  equal  to  2,500  and  defined  as  urban  by  the  U.S. 
Bureau  of  Census 


7 Rural  MSA  a place  within  an  MSA  with  a population  of  less  than  2,500  and 

defined  as  rural  by  the  U.S.  Bureau  of  Census 

8 Rural  NonMSA  a place  not  within  an  MSA  with  a population  of  less  than  2,500  and 

defined  as  rural  by  the  U.S.  Bureau  of  Census 


These  categories  are  designed  to  be  exhaustive  and  mutually  exclusive.  Every  place  in  the  50 
United  States  and  the  District  of  Columbia  is  classified  as  belonging  to  exactly  one  of  these  categories. 
The  definitions  of  MSAs  and  PMSAs,  and  their  central  cities,  are  carried  out  by  the  Office  of 
Management  and  Budget  (0MB).  0MB  Bulletin  No.  93-17  states  that  “all  agencies  that  conduct 
statistical  activities  to  collect  and  publish  data  for  Metropolitan  Areas  should  use  the  most  recent 
definitions  of  Metropolitan  Areas  established  by  0MB.”  The  definitions  used  (as  of  June  1995)  were 
those  current  at  the  time  of  the  1996  assessment.  The  definitions  of  places  and  their  populations  are 
obtained  from  the  published  results  of  the  1990  Population  Census,  as  are  the  definitions  of  Urbanized 
Areas. 


Further  details  about  the  creation  of  the  eight-category  type  of  location  variable  are  provided  in 
The  NAEP  1994  Sampling  and  Weighting  Report  (Wallace  & Rust,  1996). 
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The  variable  TOL5  was  created  by  collapsing  the  information  provided  in  the  variable  TOL8  to 
five  levels: 

1 Large  Central  City 

2 Midsize  Central  City 

3 Urban  Fringe  of  Large  City,  Urban  Fringe  of  Midsize  City,  and  Large  Town 

4 Small  Town 

5 Rural  MSA  and  Rural  NonMS A 

The  variable  TOL3  is  used  extensively  in  the  NAEP  reports.  TOL3  collapses  TOL8  to  three 

levels: 


1 Central  City  (Large  Central  City  and  Midsize  Central  City)  This  category 

includes  central  cities  of  all  MSAs.  Central  City  is  a geographic 
term  and  is  not  synonymous  with  “inner  city.” 

2 Urban  Fringe/Large  Town  (Urban  Fringe  of  Large  City,  Urban  Fringe  of  Midsize  City,  and 

Large  Town)  An  Urban  Fringe  includes  all  densely  settled 
places  and  areas  within  MSAs  that  are  classified  as  urban  by  the 
Bureau  of  Census.  A Large  Town  is  defined  as  a place  outside 
MSAs  with  a population  greater  than  or  equal  to  25,000. 

3 Rural/Small  Town  (Small  Town,  Rural  MSA,  and  Rural  NonMSA)  Rural  includes 

all  places  and  areas  with  a population  of  less  than  2,500  that  are 
classified  as  rural  by  the  Bureau  of  Census.  A Small  Town  is 
defined  as  a place  outside  MSAs  with  a population  of  less  than 
25,000  but  greater  than  or  equal  to  2,500. 


PARED  (Student’s  Report  of  Parents’  Education  Level) 

The  variable  PARED  is  derived  from  responses  to  two  questions,  B003501  and  B003601,  in  the 
student  demographic  questionnaire.  Each  student  was  asked  to  indicate  the  extent  of  his  or  her  mother’s 
education  (B003501 — How  far  in  high  school  did  your  mother  go?)  by  choosing  one  of  the  following; 

o She  did  not  finish  high  school, 
o She  graduated  from  high  school, 
o She  had  some  education  after  high  school, 
o She  graduated  from  college. 

CD  I don’t  know. 

Each  student  was  asked  to  provide  the  same  information  about  the  extent  of  his  or  her  father’s 
education  (B003601— How  far  in  high  school  did  your  father  go?)  by  choosing  one  of  the  following; 

CD  He  did  not  finish  high  school. 

CD  He  graduated  from  high  school. 

CD  He  had  some  education  after  high  school. 

CD  He  graduated  from  college. 

CD  I don’t  know. 
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The  information  was  combined  into  one  parental  education  reporting  category  (PARED)  as 
follows:  If  a student  indicated  the  extent  of  education  for  only  one  parent,  that  level  was  included  in  the 
data.  If  a student  indicated  the  extent  of  education  for  both  parents,  the  higher  of  the  two  levels  was 
included  in  the  data.  For  students  who  did  not  know  the  level  of  education  for  both  parents  or  did  not 
know  the  level  of  education  for  one  parent  and  did  not  respond  for  the  other,  the  parental  education  level 
was  classified  as  unknown.  If  the  student  did  not  respond  for  both  parents,  the  student  was  recorded  as 
having  provided  no  response. 


REGION  (Region  of  the  Country) 

Jurisdictions  were  grouped  into  four  geographical  regions— Northeast,  Southeast,  Central,  and 
West— as  shown  in  Table  B-1.  All  50  states  and  the  District  of  Columbia  are  listed.  The  part  of  Virginia 
that  is  included  in  the  Washington,  DC,  metropolitan  statistical  area  is  included  in  the  Northeast  region; 
the  remainder  of  the  state  is  included  in  the  Southeast  region. 


Table  B-1 

NAEP  Geographic  Regions 

NORTHEAST  SOUTHEAST  CENTRAL  WEST 


Connecticut 

Delaware 

District  of 

Columbia 

Maine 

Maryland 

Massachusetts 

New  Hampshire 

New  Jersey 

New  York 

Pennsylvania 

Rhode  Island 

Vermont 

Virginia 


Alabama 

Arkansas 

Florida 

Georgia 

Kentucky 

Louisiana 

Mississippi 

North  Carolina 

South  Carolina 

Tennessee 

Virginia 

West  Virginia 


Illinois 

Indiana 

Iowa 

Kansas 

Michigan 

Minnesota 

Missouri 

Nebraska 

North  Dakota 

Ohio 

South  Dakota 
Wisconsin 


Alaska 

Arizona 

California 

Colorado 

Hawaii 

Idaho 

Montana 

Nevada 

New  Mexico 

Oklahoma 

Oregon 

Texas 

Utah 

Washington 

Wyoming 


MOD  AGE  (Modal  Age) 

The  modal  age  (the  age  of  most  of  the  students  in  the  grade  sample)  for  the  fourth-grade  students 
is  age  9 (age  13  for  grade  8 and  age  17  for  grade  12).  A value  of  1 for  MODAGE  indicates  that  the 
student  is  younger  than  the  modal  age;  a value  of  2 indicates  that  the  student  is  of  the  modal  age,  a value 
of  3 indicates  that  the  student  is  older  than  the  modal  age. 
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lEP  (Individualized  Education  Program)^ 


The  variable  lEP  comes  from  the  student  booklet  cover.  A value  of  1 indicates  that  a student  has 
an  individualized  education  program,  while  a value  of  2 indicates  no  individualized  education  program. 


LEP  (Limited  English  Proficiency) 

The  variable  LEP  comes  from  the  student  booklet  cover.  A value  of  1 indicates  that  a student  is 
considered  to  have  limited  English  proficiency  while  a value  of  2 indicates  that  the  student  does  not  have 
limited  English  proficiency.  ' 


TITLEl 

The  variable  TITLEl  comes  from  the  student  booklet  cover.  A value  of  1 indicates  that  a student 
is  eligible  for  Title  1 funding  and  a value  of  2 indicates  that  the  student  is  not  eligible  for  Title  1 funding. 


SLUNCH 

SLUNCHl 

The  variable  SLUNCH  is  provided  by  Westat,  Inc.  and  is  used  to  determine  if  a student 
participates  in  the  National  School  Lunch  Program.  The  values  for  this  variable  are  as  follow: 

1 not  eligible 

2 eligible  for  reduced  price  lunch 

3 eligible  for  free  lunch 

4 no  information  available 

5 school  refused  to  provide  information 


The  variable  SLUNCHl  collapses  the  information  provided  in  the  variable  SLUNCH  to  three 

levels: 

1 eligible  for  free  or  reduced  price  lunch 

2 not  eligible  for  free  or  reduced  price  lunch 

3 no  information  available 


A student  identified  on  the  Administration  Schedule  as  a student  with  a disability  (SD)  or  an  equivalent  classification  may  be 
excluded  from  the  assessment  if;  1)  the  student  is  mainstreamed  less  than  50%  of  the  time  in  academic  subjects  and  is  Judged 
incapable  of  participating  meaningfully  in  the  assessment,  or  2)  the  Individualized  Education  Program  (lEP)  team  or  equivalent 
group  has  determined  that  the  student  is  incapable  of  participating  meaningfully  in  the  assessment.  SD/LEP  students  meeting  the 
above  criteria  should  be  assessed  if,  in  the  Judgment  of  school  staff,  they  are  capable  of  taking  the  assessment. 


SCHTYPE 


The  variable  SCHTYPE  is  provided  by  Westat,  Inc.,  and  is  used  to  determine  the  type  of  school 
that  a student  attended.  The  values  for  this  variable  are  as  follow: 

1 Public 

2 Private 

3 Catholic 

4 Bureau  of  Indian  Affairs  (BIA) 

5 Department  of  Defense  Education  Activity  (DoDEA)  schools 
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VARIABLES  DERIVED  FROM 
THE  STUDENT  AND  TEACHER  QUESTIONNAIRES 

Several  variables  were  formed  from  the  systematic  combination  of  response  values  for  one  or 
more  items  from  either  the  student  demographic  questionnaire,  the  student  background  questionnaire,  or 
the  teacher  questionnaire. 


HOMEEN2  (Home  Environment— Articles  [of  4]  in  the  Home)  - Science 


For  the  science  sample  the  variable  HOMEEN2  was  created  from  the  responses  to  student 
demographic  items  B000901  (Does  your  family  get  a newspaper  regularly?),  B000903  (Is  there  an 
encyclopedia  in  your  home?),  B000904  (Are  there  more  than  25  books  in  your  home?),  and  B000905 
(Does  your  family  get  any  magazines  regularly?).  The  values  for  this  variable  were  derived  as  follows. 

1 0-2  types  The  student  responded  to  at  least  two  items  and  answered  “yes”  to  two  or 

fewer. 


2 

3 types 

The  student 

3 

4 types 

The  student 

8 

Omitted 

The  student 

answered  “yes”  to  three  items, 
answered  “yes”  to  four  items, 
answered  fewer  than  two  items. 


HOMEEN3  (Home  Environment — Articles  [of  4]  in  the  Home)  - Mathematics 

For  the  mathematics  samples  the  variable  HOMEEN3  was  created  from  the  responses  to  student 
demographic  items  B000901  (Does  your  family  get  a newspaper  regularly?),  B000903  (Is  there  an 
encyclopedia  in  your  home?),  B008801  (How  many  books  are  in  your  home?),  collapsed  to  indicate 
whether  or  not  there  are  more  than  25  books  in  the  home),  and  B000905  (Does  your  family  get  any 
magazines  regularly?).  The  values  for  this  variable  were  derived  as  follows; 

1 0-2  types  The  student  responded  to  at  least  two  items  and  answered  yes  to  two  or 

fewer. 


2 3 types 

3 4 types 
8 Omitted 


The  student  answered  “yes”  to  three  items. 
The  student  answered  “yes”  to  four  items. 
The  student  answered  fewer  than  two  items. 
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NCOMP  (Number  of  Computer  Science  Courses  Taken) 

For  age  class  17  (long-term  trend  mathematics  and  science),  NCOMP  was  created  from 
responses  to  items  B005312  and  B005313  concerning  the  student’s  coursework  in  computer  science.  The 
values  for  NCOMP  were  derived  as  follows: 


1 0 

The  student  answered  HAVE  NOT  to  both  courses. 

2 1 

The  student  answered  HAVE  to  one  course. 

3 2 

The  student  answered  “yes”  to  both  courses. 

8 No  response 

The  student  did  not  respond  to  one  or  both  items. 

9 Mult.  & out-of-range 

The  student  filled  in  more  than  one  oval  for  both  items. 

NMATH  (Highest  Level  of  Mathematics  Courses  Taken) 

For  age  class  17,  long-term  trend  mathematics  and  science)  NMATH  was  created  from  responses 
to  items  B005301  through  B005307  concerning  the  student’s  coursework  in  mathematics.  The  values  for 
NMATH  were  derived  as  follows: 


1 Gen.  mathematics  or 
pre-algebra 

The  student  answered  HAVE  NOT  to  all  items  or 
HAVE  to  B005301  or  B005302  and  HAVE  NOT  to 
all  others. 

2 Algebra 

The  student  answered  HAVE  to  B005303  and  HAVE 
NOT  to  B005304,  B005305,  B005306,  and  B005307. 

3 Geometry 

The  student  answered  HAVE  to  both  B005303  and 
B005305  and  HAVE  NOT  to  B005304,  B005306,  and 
B005307. 

4 Algebra  2 

The  student  answered  HAVE  to  B005304  or  B005306 
but  HAVE  NOT  to  B005307. 

5 Calculus 

The  student  answered  HAVE  to  B005307. 

6 Something  else 

Any' other  response  combination 

8 No  response 

The  student  did  not  respond  to  any  item. 

.y  . 
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NSCI  (Highest  Level  of  Science  Courses  Taken) 

For  age  class  17,  (long-term  trend  mathematics  and  science)  NSCI  was  created  from  responses  to 
items  B005308  through  B00531 1,  which  concerned  the  student’s  coursework  in  science.  The  values  for 
NSCI  were  derived  as  follows: 


1 No  biology 

The  student  answered  HAVE  NOT  to  all  items  or  HAVE  to 
B005308  and  other  than  HAVE  to  all  other  items. 

2 Biology 

The  student  answered  HAVE  to  B005309  and  other  than 
HAVE  to  both  B0053 10  and  B0053 1 1 . 

3 Chemistry 

The  student  answered  HAVE  to  both  B005309  and  B005310 
and  other  than  HAVE  to  B0053 1 1 . 

4 Physics 

The  student  answered  HAVE  to  B005309,  B005310,  and 
B005311. 

5 Something  else 

Any  other  response  combination 

8 No  response 

The  student  answered  none  of  the  items. 

VARIABLES  DERIVED  FROM  COGNITIVE  ITEMS 


BKSCOR  (Booklet-Level  Score) 

The  booklet-level  score  is  a student-level  score  based  on  the  sum  of  the  number  correct  for 
dichotomous  items  plus  the  sum  of  the  scores  on  the  polytomous  items,  where  the  score  for  a polytomous 
item  starts  from  0 for  the  unacceptable  category.  Thus,  for  a 4-point  extended  constructed-response  item, 
scores  of  “no  response”,  “off-task”,  and  “unsatisfactory”  are  assigned  an  item  score  of  0.  Scores  of 
“partial,”  “essential,”  and  “extensive”  are  assigned  item  scores  of  1,  2,  and  3,  respectively.  The  score  is 
computed  based  on  all  cognitive  items  in  a student’s  assessment  booklet. 


LOGIT  (Logit  Percent  Correct  Within  Booklet) 


In  order  to  compute  the  LOGIT  score,  a percent  correct  within  booklet  was  first  computed.  This 
score  was  based  on  the  ratio  of  the  booklet  score  (BKSCOR)  over  the  maximum  booklet  score.  The 
percent  correct  score  was  set  to  .0001  if  no  items  were  answered  correctly;  if  BKSCOR  equaled  the 
maximum  booklet  score,  the  percent  correct  score  was  set  to  .9999.  A logit  score,  LOGIT,  was  calculate 
for  each  student  by  the  following  formula: 


LOGIT  = In 


PCTCOR 

l-PCTCOR. 


A logit  score,  LOGIT,  was  calculated  within  booklet  for  each  student  by  the  following  formula: 
LOGIT  was  then  truncated  to  a value  x,  such  that  -3  ^ x ^3.  After  computing  LOGIT  for  each  student, 
the  mean  and  standard  deviation  were  calculated  for  each  booklet  as  the  first  step  in  standardizing  the 
logit  scores.  The  standardized  logit  score,  ZLOGIT,  was  then  calculated  for  each  student  by  the  following 
formula: 


ZLOGIT  = 


LOGIT -mean  logit 
standard  deviation 
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NORMIT  (Norinit  Gaussian  Score) 

SCHNORM  (School-Level  Mean  Gaussian  Score) 

The  normit  score  is  a student-level  Gaussian  score  based  on  the  inverse  normal  transformation  of 
the  mid-percentile  rank  of  a student’s  number-correct  booklet  score  within  that  booklet.  The  normit 
scores  were  used  to  decide  collapsing  of  variables,  finalize  conditioning  coding,  and  check  the  results  of 
scaling. 


The  number  correct  is  based  on  the  number  of  dichotomous  items  answered  correctly  plus  the 
score  obtained  on  extended  constructed-response  items.  The  mid-percentile  rank  is  based  on  the  formula: 

CF(0  + CF(/-1) 

2N 

where  CF(/)  is  the  cumulative  frequency  at  i items  correct  and  N is  the  total  sample  size.  If  / = 0 then 

CF{0)  + ^^ 

W 

A school-level  normit,  SCHNORM,  was  also  created;  this  was  the  mean  normit  across  all  reading 
booklets  administered  in  a school.  These  school-level  mean  normit  scores  were  used  in  conditioning 
procedures  to  take  into  account  differences  in  school  proficiency.  For  each  school,  the  weighted  mean  of 
the  logits  for  the  students  in  that  school  was  calculated.  Each  student  was  then  assigned  that  mean  as  his 
or  her  school-level  mean  logit  score  value. 
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VARIABLES  RELATED  TO  PROFICIENCY  SCALING 


Proficiency  Score  Variables 

Item  response  theory  (IRT)  was  used  to  estimate  average  proficiency  for  the  nation  and  for 
various  subpopulations,  based  on  students’  performance  on  the  set  of  cognitive  items  they  received.  IRT 
provides  a common  scale  on  which  performance  can  be  reported  for  the  nation  and  subpopulations,  even 
when  all  students  do  not  answer  the  same  set  of  questions.  This  common  scale  makes  it  possible  to  report 
on  relationships  between  students’  characteristics  (based  on  their  responses  to  the  background  questions) 
and  their  overall  performance  in  the  assessment. 

A scale  ranging  from  1 to  500  or  from  1 to  300  was  created  to  report  performance  for  each 
content  area  or  strand.  A composite  scale  was  created  based  on  a weighted  average  of  scales,  where  the 
weight  for  each  content  area  or  strand  was  proportional  to  the  relative  importance  assigned  to  the  content 
area  as  specified  in  the  mathematics  and  science  objectives. 

Scale  proficiency  estimates  were  obtained  for  all  students.  The  NAEP  methods  use  random 
draws  (plausible  values)  from  estimated  proficiency  distributions  to  compute  population  statistics. 
Plausible  values  are  not  optimal  estimates  of  individual  proficiency;  instead,  they  serve  as  intermediate 
values  to  be  used  in  estimating  population  characteristics.  Chapter  11  provides  further  details  on  the 
computation  and  use  of  plausible  values.  Chapters  12-18  provide  additional  information  as  appropriate  to 
each  sample/subject  area. 

The  proficiency  score  (plausible  value)  variables  are  provided  on  the  student  data  files  for  each  of  the 
scales  and  are  named  as  shown  in  Table  B-2. 


Table  B-2 

Scaling  Variables  for  the  1996  National  Assessment  Samples 


Sample 

Scale 

Data  Variables 

Mathematics  Main 

Number  Sense,  Properties,  and 

Operations 

Measurement 

Geometry  and  Spatial  Sense 
Data  Analysis,  Statistics,  and 
Probability 

Algebra  and  Functions 
Composite 

MRPSll  toMRPSlS 

MRPS21  to  MRPS25 
MRPS31  to  MRPS35 
MRPS41  to  MRPS45 

MRPS51  to  MRPS55 
MRPCMl  to  MRPCM5 

Science  Main 

Physical  Science 
Earth  Science 
Life  Science 
Composite 

SRPSll  to  SRPS15 
SRPS21  to  SRPS25 
SRPS31  to  SRPS35 
SRPCMl  to  SRPCM5 

Reading  Long-Term  Trend 

Univariate 

REDVALl  to  REDVAL5 

2 2 

Writing  Long-Term  Trend 

Univariate 

WRPSCTl  to  WRPSCT5 

Mathematics  Long-Term  Trend 

Univariate 

MRPSCTl  to  MRPSCT5 

Science  Long-Term  Trend 

Univariate 

SRPSCTl  to  SRPSCT5 
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SMEANx,  SMNxl 
SRANKx,  SRNKxl 
SRNK3X,  SRK3xl 
where  x = M or  S 


(School  Mean  Score  Using  First  Plausible  Value) 

(School  Rank  Using  First  Plausible  Value) 

(Top,  Middle,  Bottom  Third  Using  First  Plausible  Value 
for  Science  or  Mathematics) 


A mean  composite  score  (SMEANx  on  the  student  files,  SMNxl  on  the  school  files)  was 
calculated  using  the  first  composite  plausible  value  for  each  school  within  each  grade/subject  area.  The 
mean  composite  score  was  based  on  the  values  from  the  scaling  variable  xRPCMl  and  was  calculated 
using  the  students’  sampling  weights.  The  schools  were  then  ordered  from  highest  to  lowest  mean  score 
(SRANKx  on  the  student  files,  SRNKxl  on  the  school  files)  within  a sample  using  school-level  weights 
— the  school  with  the  highest  mean  score  was  given  a ranking  of  1,  and  the  school  with  the  lowest  mean 
score  was  given  a ranking  equal  to  the  number  of  schools  in  the  sample. 

These  variables  were  then  used  in  partitioning  the  schools  within  the  national  grade  sample  into 
three  groups  (top  third,  middle  third,  and  bottom  third)  based  on  their  ranking  (SRNK3x  on  the  student 
files,  SRK3xl  on  the  school  files). 


SMNxn  (School  Mean  Score  Using  Plausible  Values  2 Through  5) 

SRNKxn  (School  Rank  Using  Plausible  Values  2 Through  5) 

SRK3xn  (Top,  Middle,  Bottom  Third  Using  Plausible  Values  2 Through  5) 

SMNRPn  (School  Mean  Score  Using  Plausible  Values  2 Through  5,  Public  Schools 

Only) 

School  ranking  results  presented  in  the  1996  NAEP  reports  are  based  on  the  first  plausible  value. 
However,  since  there  are  four  additional  estimates  of  proficiency  (plausible  values)  for  each  student, 
school  ranking  data  were  also  created  for  those  estimates.  These  school  rank  values  were  created  using 
the  same  procedures  described  above,  substituting  proficiency  variables  xRPCM2  through  xRPCMS  to 
compute  the  results.  In  the  variable  names,  n denotes  the  plausible  value  2,  3,  4,  or  5. 
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QUALITY  EDUCATION  DATA  VARIABLES  (QED) 


The  data  files  contain  several  variables  obtained  from  information  supplied  by  Quality  Education 
Data,  Inc.  (QED).  QED  maintains  and  annually  updates  lists  of  schools  showing  grade  span,  total 
enrollment,  instructional  dollars  per  pupil,  and  other  information  for  each  school.  These  data  variables 
are  retained  on  both  the  school  and  student  files  and  are  identified  in. the  data  layouts  by  “(QED)”  in  the 
SHORT  LABEL  field. 

Most  of  the  QED  variables  are  defined  sufficiently  in  the  data  codebooks.  Explanations  of  others 
are  provided  below. 

ORSHPT  is  the  Orshansky  Percentile,  an  indicator  of  relative  wealth  that  specifies  the 
percentage  of  school-age  children  in  a district  who  fall  below  the  poverty  line. 

IDP  represents,  at  the  school  district  level,  dollars  per  student  spent  for  textbooks  and 
supplemental  materials.  The  range  code  for  instructional  dollars  spent  per  pupil  excluding  teacher 
salaries  are: 

0 = Unclassified 

1 = Under  $10 

2 = $10-49 

3 = $50-99 

4 = $100-149 

5 = $150-299 

6 = $300-399 

7 = $400-499 

8 = $500-999 

9 = $1000  + 

ADULTED  indicates  whether  or  not  adult  education  courses  are  offered  at  the  school  site: 

URBAN  defines  the  school’s  urbanization:  urban  (central  city);  suburban  (area  surrounding 
central  city,  but  still  located  within  the  counties  constituting  the  metropolitan  statistical  area);  or  rural 
(area  outside  any  metropolitan  statistical  area). 
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Appendix  C 


CONDITIONING  VARIABLES 
AND  CONTRAST  CODINGS 


This  appendix  contains  information  about  the  conditioning  variables  used  in  scaling/plausible 
value  estimation  for  the  1996  NAEP  assessment.  The  initial  step  in  construction  of  conditioning  variables 
involves  forming  primary  student-based  vectors  of  response  data  from  answers  to  student,  teacher,  and 
school  questionnaires,  demographic  and  background  data  such  as  supplied  by  Westat,  Inc.,  and  other 
student  information  known  prior  to  scaling.  The  initial  conditioning  vectors  concatenate  this  student 
background  information  into  a series  of  identifying  “contrasts”  comprising: 

1.  Categorical  variables  derived  by  expanding  the  response  options  of  a 
questionnaire  variable  into  a binary  series  of  one-degree-of-freedom 
“dummy”  variables  or  contrasts,  (these  form  the  majority  of  each  student 
conditioning  vector); 

2.  Questionnaire  or  demographic  variables  that  possess  ordinal  response 
options,  such  as  number  of  hours  spent  watching  television,  which  are 
included  as  linear  and/or  quadratic  multi-degree-of-freedom  contrasts; 

3.  Continuous  variables,  such  as  student  logit  scores  based  on  percent  correct 
values,  included  as  contrasts  in  their  original  form  or  a transformation  of 
their  original  form,  and; 

4.  Interactions  of  two  or  more  categorical  variables  forming  a set  of  orthogonal 
one-degree-of-freedom  dummy  variables  or  contrasts. 

This  appendix  gives  the  specifications  used  for  constructing  the  conditioning  variables.  Table  C- 
1 defines  the  information  provided  for  each  main  sample  variable.  Table  C-2  provides  a summary  of  the 
mathematics  conditioning  variables  specifications  that  are  contained  in  the  remainder  of  this  appendix. 
Table  C-3  provides  a summary  of  the  science  conditioning  variables  specifications  that  are  contained  in 
the  remainder  of  this  appendix.  Conditioning  variable  data  specific  to  each  subject  area  is  shown  for 
main  mathematics  in  Table  C-4  and  for  science  in  Table  C-5.  Similar  information  for  long-term  trend 
samples  is  given  in  Tables  C-6,  C-7,  C-8,  and  C-9  respectively  for  reading,  mathematics,  science,  and 
writing. 


As  described  in  Chapter  9,  the  linear  conditioning  model  employed  for  the  estimation  of 
plausible  values  did  not  directly  use  the  conditioning  variable  specifications  listed  in  this  appendix.  To 
eliminate  inherent  instabilities  in  estimation  encountered  when  using  a large  number  of  correlated 
variables,  a principal  component  transformation  of  the  correlation  matrix  obtained  from  the  conditioning 
variable  contrasts  derived  according  to  these  primary  specifications  was  performed.  The  principal 
components  scores  based  on  this  transformation  were  used  as  the  predictor  variables  in  estimating  the 
linear  conditioning  model.  The  proportions  of  variance  of  the  conditioning  contrast  accounted  for  by  the 
principal  components  are  given  for  each  age/grade  level  in  Tables  C-10,  C-1 1,  and  C-12  for  mathematics, 
and  Tables  C-13,  C-14,  and  C-15  for  science. 
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Table  C-1 

Description  of  Specifications  Provided  for  Each  Conditioning  Variable 


Title 

CONDITIONING  ID 


DESCRIPTION 

GRADES/ASSESSMENTS 


CONDITIONING  VAR 

LABEL 

NAEPID 

TYPE  OF  CONTRAST 


TOTAL  NUMBER  OF 
SPECIFIED  CONTRASTS 


NUMBER  OF  INDEPENDENT 
CONTRASTS 


Description 

A unique  eight-character  ID  assigned  to  identify  each  conditioning 
variable  corresponding  to  a particular  background  or  subject  area 
question  within  the  entire  pool  of  conditioning  variables.  The  first 
four  characters  identify  the  origin  of  the  variable:  BACK  (background 
questionnaire),  READ  (student  reading  questionnaire),  SCHL  (school 
questionnaire),  TCHR  (background  part  of  teacher  questionnaire),  and 
TSUB  (subject  classroom  part  of  teacher  questionnaire).  The  second 
four  digits  represent  the  sequential  position  within  each  origin  group. 

A short  description  of  the  conditioning  variable. 

Three  characters  identifying  assessment  (“S”  for  state,  “N”  for 
national)  and  grade  (04,  08,  and  12)  in  which  the  conditioning 
variable  was  used. 

A descriptive  eight-character  label  identifying  the  conditioning 
variable. 

The  seven-character  NAEP  database  identification  for  the 
conditioning  variable. 

The  type  of  conditioning  variable.  “CLASS”  identifies  a categorical 
conditioning  variable  and  “SCALE”  identifies  continuous  or  quasi- 
continuous  conditioning  variables.  “INTERACTION”  identifies  a set 
of  orthogonal  contrasts  formed  from  two  or  more  “CLASS”  variables. 
“OTHER”  conditioning  variables  do  not  fall  into  any  of  the  above 
types. 

Each  conditioning  variable  forms  a set  of  one  or  more  contrasts.  For 
each  valid  response  value  of  conditioning  variable  a contrast  must  be 
defined.  One  or  more  response  values  may  be  collapsed  together  to 
form  one  contrast.  The  number  of  response  value  “sets”  of  a 
conditioning  variable  forming  a unique  contrast  is  the  value  given  in 
this  field. 

The  number  of  degree  of  freedom  in  a set  of  contrasts  formed  from  a 
conditioning  variable.  For  a categorical  conditioning  variable  this 
number  would  be  the  number  of  response  options  minus  one  if  each 
response  option  formed  its  own  unique  contrast. 


479 


451 


Table  C-2 

Summary  Table  of  the  1996  Mathematics  Assessment  Conditioning  Variable  Specifications 


ERIC 


0) 

'S 
2 ^ 
a 

'S 

p 00 

O 


2^ 

o 


o 

•^3 

A 

*c 


u 

H 


fi 

*3 

o 

la 

s 

o 

U 


oo 

XXXXXXXXXXXXXXXXXXXXXXXXXX  I XXXXXXXXXXXXXXXX  I X XX  X 

I I I 


XXXXXXXXXXXXXXXXXXXXXXXXXXXXX  I xxxx 


I I I I I I I I 


I I I I I 


Q 

X 

> 

H 

X 

X 

Q 

W 

X 

X 

X 

H 

w 

tsi 

H 

< 

iJ 

o 

o 

Q cn  1X4  cn  1X4 

^ ot:  o a:  o 

H W w 
Eh  pc:  Eh  pc: 

w w w w 


X P 

q X P 

q u p 

q Eh  w p 

9 u 

3 X 5 

e:  X S 

3 < X S 

3 X 

w E 

q w E 

q w r 

q X w E 

q w 

^ 7 

5 w s 

3 § cn  E 

5 w 

o 

PQ 

fr* 


tH  tH  tH  tH  tH  tH 

O 00  o o o o o 

(N  'O’  00  00  \£>  \£>  \£> 

n (N  iH  iH  \£>  \£>  \£> 

o o o o o o o 

o o o o o o o 

pq  pq  pq  pq  pq  pq  pq 

X Eh  Eh 


iH  iH  \£>  in 
O O O O 

\£>  r-  o o o 
m in  ’O'  o iH 
o o o o o 
o o o o o 

pq  pq  w O J 

Eh  Eh  Eh  J h) 


tH 

X 

tH 

tH 

tH 

tH 

tH 

tH  (N 

tH 

tH 

tH 

tH 

tH 

X 

O 

X 

04 

tH 

X o 

O 

o 

O 

o 

o 

o S 

o 

o 

o 

o 

o 

tH 

Q 

o 

>H 

u 

O (N 

a\ 

00 

00 

VO 

VO 

VO  X 

VO 

r- 

o 

o 

VO 

u 

X o 

m 

00 

in 

pp 

M 

m 

00 

tH 

tH 

VO 

VO 

VO  W 

in 

in 

00 

w 

u 

S 

o 

U 

S 

o 

X 

04 

04 

5 

o 

o 

O 

O 

o 

o 

o S 

o 

o 

o 

o 

o 

X 

w 

P3 

o 

O 

O 

O 

X 

o 

X 

X 

H 

J o 

o 

O 

O 

o 

o o O 

o 

o 

o 

o 

o 

n 

Q 

Q 

ffl 

Eh 

Eh 

Q 

04 

K W 

H 

i-q 

Eh 

w 

m 

X 

X 

X 

X 

X 

X X 

X 

X 

w 

X 

X 

tH 

r>j 

m 

in 

l£> 

00 

m 

o 

tH 

CO 

in 

VO 

00 

a\ 

o 

tH  CM 

ro 

in 

VO 

r- 

o 

o 

o 

o 

o 

O 

o 

o 

o 

tH 

tH 

tH 

tH 

tH 

tH 

tH 

tH 

tH 

tH 

CM 

CM  CM 

CM 

CM 

CM 

CM 

CM 

o 

o 

o 

o 

o 

O 

o 

o 

o 

O 

O 

o 

O 

O 

o 

O 

O 

O 

O 

o 

O O 

o 

O 

o 

o 

o 

o 

o 

o 

o 

o 

O 

o 

o 

o 

O 

O 

o 

O 

O 

o 

O 

O 

O 

O 

o 

O O 

O 

O 

o 

o 

o 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X X 

X 

X 

X X 

X 

X 

X 

X X 

X 

X 

X 

X X 

o 

o 

o 

O 

o 

o 

o 

u 

u 

o 

o 

o o 

u 

o 

u u 

u 

o 

o 

o u 

u 

u 

u.  o o 

< 

< 

< 

< 

< 

< 

< 

< 

< 

< 

< < 

< 

< 

< 

< 

< 

< 

< < < 

< 

< 

< < 

< 

CQ 

CQ 

m 

(Q 

(Q 

CQ 

(Q 

m 

m 

n 

n 

n 

m 

n 

X 

X 

X 

X 

X 

X X 

X 

X 

X 

X 

X 

oo(Nr^tHiHO\roro^^ininvoio  r^r^oooo 

voi— itHOO^rorororororororo  rorororo 

1— iinrorororororororo  rorororo 

OOOiHiHCNOO.OOOOOO  OOOO 

OOOOOOOOOOOOOO  OOOO 

OOOOOOOOOOOOOO  OOOO 

OUDqpqpqDqDqUDqDqDqDqDqDq  Dqpqpqpq 

i-:]xxehehxxxxxxxxx  xxxx 


rHrHrHrHrHrHrHrHCNCNrnrn^^  W LO  LO  VD  VD 


OOOO 


P500000000000000 
O rO  rO  ^ tH  tH  in  tHtHtHtHtHtHtHtHrfJiHiHiHt-H 
goor^r^rHrHoor^r^r^r^r^r^r^r^EHr^r^r^r“ 
Xooooooooooooooehoooo 


^in'^r^OOtTkOrHrHCNrO^in 


soooooooooooooomoooo 


o 

CO 


lULi 

§ 


o 

C3 


(N 


Table  C-2  (continued) 
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Table  C-4 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  OVERALL  (@ 

conditioning  variable  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
conditioning  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  MALE  (1 

002  FEMALE  (2 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  WHI/Al/0  (1,5,6 

002  BLACK  (2 

003  HISPANIC  (3 

004  ASIAN  (4 


BACKOOOl 
GRAND  MEAN 

N04,  S04,  NOS,  SOS,  N12 
OVERALL  . 

BKSER 

OTHER 

) 1 

BACK0002 
DERIVED  SEX 

N04,  S04,  NOS,  SOS,  N12 

GENDER 

DSEX 

CLASS 

) 0 
) 1 

BACK0003 

DERIVED  RACE/ ETHNICITY 

N04,  S04,  NOS,  SOS,  N12 

RACE/ETH 

DRACE 

CLASS 

) 000 

) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

GRAND  MEAN 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

MALE 

FEMALE 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  4 

NUMBER  OF  INDEPENDENT  CONTRASTS:  3 

RACE/ETHNiCiTY:  WHITE,  AMERICAN  INDIAN /ALASKAN 

NATIVE,  OTHER,  MISSING,  UNCLASSIFIED 
RACE/ETHNICITY:  BLACK 

RACE/ETHNICITY:  HISPANIC 

RACE/ETHNICITY:  ASIAN 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0004 

IF  HISPANIC,  WHAT  IS  YOUR  HISPANIC  BACKGROUND? 
N04,  S04,  NOS,  SOS,  N12 
HISPANIC 
B003101 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001  NOT  HISP  (1  ) 0000 

002  MEXICAN  (2  ) 1000 

003  PUER  RIC  (3  ) 0100 

004  CUBN.OTH  (4,5  ) 0010 

005  HISP-?  (M  ) 0001 


HISPANIC 

HISPANIC 

HISPANIC 

HISPANIC 

HISPANIC 


not  HISPANIC 

MEXICAN,  MEXICAN  AMERICAN,  CHICANO 
PUERTO  RICAN 
CUBAN , OTHER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0005 
MSA/ NON-MSA 

N04,  S04,  NOS,  SOS,  N12 

MSANAT 

TOLS 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  MSA  (1,2, 3, 4,7, 9)  0 

002  NON  MSA  (5,6,S  ) 1 


MSA 

NON-MSA 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  BIG  CTY5  (1 

002  MID  CTY5  (2,M 

003  FR/BTWN5  (3 

004  SML  TWN5  (4 

005  RURAL5  (5 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0006 

TYPE  OF  LOCALE  (5  CATEGORIES) 

N04,  S04,  NOS,  SOS,  N12 

TOLS 

TOLS 

CLASS 


BACK0007 

DESCRIPTION  OF  COMMUNITY 

N04.  S04,  NOS,  SOS.  N12 

DOC 

DOC 

CLASS 


) 0000 
) 1000 
) 0100 

) 0010 
).  0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  5 

NUMBER  OF  INDEPENDENT  CONTRASTS:  4 

TOLS:  LARGE  CITY 

TOLS:  MID-SIZE  CITY 

TOLS:  URBAN  FRINGE  OF  LARGE  CITY.  URBAN  FRINGE  OF 

MID-SIZE  CITY 

TOLS : SMALL  TOWN 

TOLS:  RURAL  (MSA  AND  NON-MSA) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  BIG  CITY  (1  ) 000 

002  URBAN  FR  (2  ) 100 

003  MED  CITY  (3.9.M  ) 010 

004  SM  PLACE  (4  ) 001 


DOC: 

BIG  CITY 

DOC: 

URBAN  FRINGE 

DOC: 

MEDIUM  CITY 

DOC: 

SMALL  PLACE 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0008 

PARENTS ' HIGHEST  LEVEL 

N04,  S04.  NOS.  SOS.  N12 

PARED 

PARED 

CLASS 


OF  EDUCATION 


TOTAL  number  OF  SPECIFIED  CONTRASTS: 
number  of  INDEPENDENT  CONTRASTS: 


5 

4 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


001  < HS  (1 

002  HS  GRAD  (2 

003  POST  HS  (3 

004  COL  GRAD  (4 

005  PARED-?  (5,M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  N EAST  (1 

002  S EAST  (2 

003  CENTRAL  (3 

004  WEST  (4,5 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0009 

REGION  OF  THE  COUNTRY 

N04,  NOS,  N12 

REGION 

REGION 

CLASS 

) 000 
) 100 
) 010 
) 001 

BACKOOlO 
SCHOOL  TYPE  (PQ) 

N04,  S04,  NOS,  SOS,  N12 

SCHTYPE 

SCHTYPE 

CLASS 


PARED:  LESS  THAN  HIGH  SCHOOL 

PARED:  HIGH  SCHOOL  GRADUATE 

PARED:  POST  HIGH  SCHOOL 

PARED : COLLEGE  GRADUATE 

PARED:  MISSING,  I DON'T  KNOW 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  4 

NUMBER  OF  INDEPENDENT  CONTRASTS:  3 

REGION:  NORTHEAST 

REGION:  SOUTHEAST 

REGION:  CENTRAL 

REGION:  WEST,  TERRITORIES  (NONE) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  3 

NUMBER  OF  INDEPENDENT  CONTRASTS : 2 


001 

PUBLIC 

(1 

) 00 

002 

PRIVATE 

(2, 4, 5,M 

) 10 

003 

CATHOLIC 

(3 

) 01 

SCHOOL  TYPE:  PUBLIC 

SCHOOL  TYPE:  PRIVATE,  BUREAU  OF  INDIAN  AFFAIRS, 

DEPARTMENT  OF  DEFENSE,  MISSING 
SCHOOL  TYPE : CATHOLIC 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


BACKOOll 

INDIVIDUALIZED  EDUCATION  PROGRAM 

N04,  S04,  NOS,  SOS,  N12 

lEP 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  lEP-YES  (1  ) 0 

002  lEP-NO  (2  ) 1 


lEP:  YES 

lEP;  NO 


2 

1 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


BACK0012 

LIMITED  ENGLISH  PROFICIENCY 

N04,  S04,  NOS,  SOS,  N12 

LEP 

LEP 

CLASS 


001  LEP-YES  (1  ) 0 

002  LEP-NO  (2  ) 1 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 

001  CHAPl-Y  (1 

002  CHAPl-N  (2 


BACK0013 

CHAPTER  1 (BOOK  COVER) 

N04,  S04,  NOS,  SOS,  N12 

CHAPTERl 

CHAPl 

CLASS 

) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

LEP;  YES 
LEP;  NO 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

CHAPTER  1 ; YES 
CHAPTER  1 ; NO 


2 

1 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


BACK0014 


DO  YOU  RECEIVE  A FREE  OR  REDUCE 

N04,  S04,  NOS,  SOS,  N12 

LUNCH 

SLUNCH 

CLASS 


-PRICE  LUNCH? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  NOT  ELIG  (1  ) 0000 

002  RED  PRIC  (2  ) 1000 

003  FREE  (3  ) 0100 

004  INFO  N/A  (4,M  ) 0010 

005  SCH/REF  (5  ) 0001 


LUNCH  PROGRAM; 
LUNCH  PROGRAM; 
LUNCH  PROGRAM; 
LUNCH  PROGRAM; 
LUNCH  PROGRAM; 


NOT  ELIGIBLE 
REDUCED  PRICE 
FREE 

INFO  NOT  AVAILABLE 
SCHOOL  REFUAL 


5 

4 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


BACK0015 

HOW  OFTEN  DO  THE  PEOPLE  IN 

N04,  S04,  NOS,  SOS,  N12 

HOME LANG 

B003201 

CLASS 


YOUR  HOME  SPEAK  A LANGUAGE  OTHER  THAN  ENGLISH? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


4 

3 


001 

HL-NEVER 

(1 

) 000 

002 

HL-SOME 

(2 

) 100 

003 

HL-ALWAY 

(3 

) 010 

004 

HL-? 

(M 

) 001 

HOMELAND ; 
HOMELAND ; 
HOMELAND; 
HOMELAND ; 


NEVER 

SOMETIMES 

ALWAYS 

MISSING 


CONDITIONING  VARIABLE  ID:  BACK0016 

DESCRIPTION;  DO  YOU  HAVE  YOUR  OWN  STUDY  DESK  OR  TABLE  AT  HOME? 


ERIC 
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Table  C-4  (continued) 
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GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


N04,  S04,  NOS,  SOS,  N12 

B00S901 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  4 

NUMBER  OF  INDEPENDENT  CONTRASTS:  3 


001  B00S901Y  (01  ) 000 

002  B00S901N  (02  ) 100 

003  B00S901I  (03  ) 010 

004  B00S901M  (M  ) 001 


YES 

NO 

IDK25  (FILL  1 SHELF) 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0017 

HOW  MUCH  TELEVISION  DO  YOU 

N04,  S04,  NOS,  SOS,  N12 

TVWATCHL 

BOOISOI 

LINEAR 


USUALLY  WATCH  EACH  DAY?  (LINEAR) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


7 

1 


001 

TVLIN-0 

(1  ) 0 

002 

TVLIN-1 

(2  ) 1 

003 

TVLIN-2 

(3  ) 2 

004 

TVLIN-3 

(4,M  ) 3 

005 

TVLIN-4 

(5  ) 4 

006 

TVLIN-5 

(6  ) 5 

007 

TVLIN-6 

(7  ) 6 

TV  WATCHING  (LINEAR)  (0  TO  6+  HOURS  PER  DAY) 
TV  WATCHING  (LINEAR) 

TV  WATCHING  (LINEAR) 

TV  WATCHING  (LINEAR) 

TV  WATCHING  (LINEAR) 

TV  WATCHING  (LINEAR) 

TV  WATCHING  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACKOOIS  . 

HOW  MUCH  TELEVISION  DO  YOU  USUALLY  WATCH  EACH  DAY? 


(QUADRATIC) 


N04,  S04,  NOS,  SOS,  Nl2 

TVWATCHQ 

BOOISOI 

QUADRATIC 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


1 

1 


001  TV-QUAD  (1-7,M=4 


1.0  + -'2.0*X  + 1.0*X**2  TV  WATCHING  (QUADRATIC) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0019 

HOMEWORK  ASSIGNED?:  BASED  ON  TIME  SPENT  ON  HOMEWORK  EACH  DAY. 

N04,  S04,  NOS,  SOS,  Nl2 
HWASSIGN 


B006601 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  HW-MISS  (M 

002  HW-NO  (1 

003  HW-YES  (2-5 


) 00 
) 10 
) 01 


HOMEWORK  ASSIGNED?:  MISSING 

HOMEWORK  ASSIGNED?:  NO 

HOMEWORK  ASSIGNED?:  YES 


3 

2 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0020 

HOW  MUCH  TIME  DO  YOU  USUALLY 

N04,  S04,  NOS,  SOS,  Nl2 

HOMEWRKL 

B006601 

LINEAR 


SPEND  ON  HOMEWORK  EACH  DAY?  (LINEAR) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


4 

1 


001 

HWLIN-0 

(1. 2,M 

) 0 

002 

HWLIN-1 

(3 

) 1 

003 

HWLIN-2 

(4 

) 2 

004 

HWLIN-3 

(5 

) 3 

HOMEWORK  (LINEAR): 
MISSING 

HOMEWORK  (LINEAR) : 
HOMEWORK  (LINEAR) : 
HOMEWORK  (LINEAR) : 


DON'T  HAVE  ANY,  DON'T  DO  ANY, 

1/2  HOUR  OR  LESS 
1 HOUR 

MORE  THAN  1 HOUR 


SPEND  ON  HOMEWORK  EACH  DAY  (QUADRATIC) 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  HWQUAD-0  (1,2,M 

002  HWQUAD-1  (3 

003  HWQUAD-2  (4 

004  HWQUAD-3  (5 


BACK0021 

HOW  MUCH  TIME  DO  YOU  USUALLY 

N04,  S04,  NOS,  SOS,  Nl2 

HOMEWRKQ 

B006601 

SCALE 

) 0 

) 1 
) 4 
) 9 


HOMEWORK  (QUADRATIC) : 
ANY,  MISSING 
HOMEWORK  (QUADRATIC) : 
HOMEWORK  (QUADRATIC) : 
HOMEWORK  (QUADRATIC) : 


DON'T  HAVE  ANY,  DON'T  DO 

1/2  HOUR  OR  LESS 
1 HOUR 

MORE  THAN  1 HOUR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0022 

NUMBER  OF  ITEMS  IN  THE  HOME 


(DERIVED) 

N04,  S04,  NOS,  SOS,  Nl2 

HOMEITMS 

HOMEEN2 

CLASS 


(NEWSPAPER,  >25  BOOKS,  ENCYCLOPEDIA,  MAGAZINES) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  HITEM<=2  (l.M  ) 00 

002  HITEM=3  (2  ) 10 

003  HITEM=4  (3  ) 01 


ITEMS  IN  HOME 
ITEMS  IN  HOME 
ITEMS  IN  HOME 


ZERO  TO  TWO 
THREE  ITEMS 
FOUR  ITEMS 


ITEMS.  MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


BACK0023 

DOES  MOTHER  OR  STEPMOTHER  LIVE  AT  HOME  WITH  YOU? 

N04,  S04,  NOS,  SOS,  N12 

MOM0HOME 


528 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


NAEP  ID:  B005601 

TYPE  OF  CONTRAST:  CLASS 

001  MOMHOM-Y  (1  ) 00 

002  MOMHOM-N  (2  ) 10 

003  MOMHOM-?  (M  ) 01 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  B005701 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


MOTHER  AT  HOME 
MOTHER  AT  HOME 
MOTHER  AT  HOME 


BACK0024 

DOES  FATHER  OR  STEPFATHER  LIVE  AT  HOME  WITH  YOU? 

N04,  S04,  NOS,  SOS,  N12 

DAD0HOME 


YES 

NO 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  DADHOM-Y 

(1 

) 00 

002  DADHOM-N 

(2 

) 10 

003  DADHOM-? 

(M 

) 01 

CONDITIONING 

VARIABLE  ID: 

BACK0025 

DESCRIPTION: 

HOW  MANY 

grades /ASSESSMENTS : 

N04,  S04 

CONDITIONING 

VAR  LABEL: 

SCH  MISS 

NAEP  ID: 

S004001 

TYPE  OF  CONTRAST: 

CLASS 

001  MlSS->2 

(3,4,5,M 

> 0 

002  MlSS-2< 

(1,2 

) 1 

CONDITIONING 

VARIABLE  ID: 

BACK0026 

DESCRIPTION: 

HOW  LONG 

GRADES/ASSESSMENTS : 

N04,  S04, 

CONDITIONING 

VAR  LABEL: 

YRSINUSA 

NAEP  ID: 

BOOSOOl 

TYPE  OF  CONTRAST: 

CLASS 

001  ALL  MY  L 

(1  ) 

0000 

002  USA  >5 

(2  ) 

1000 

003  USA  3-5 

(3  ) 

0100 

004  USA  <3 

(4  ) 

0010 

005  USA-? 

(M  ) 

0001 

CONDITIONING 

VARIABLE  ID: 

BACK0027 

DESCRIPTION: 

HOW  MANY . 

GRADES /ASSESSMENTS : 

N04,  S04 

CONDITIONING 

VAR  LABEL: 

STGRADE4 

NAEP  ID: 

B007601 

TYPE  OF  CONTRAST: 

CLASS 

001  STGRD<1 

(1,M  ) 

00 

002  STGRDl-2 

(2  ) 

10 

003  STGRD3> 

(3  ) 

01 

FATHER  AT  HOME 
FATHER  AT  HOME 
FATHER  AT  HOME 


YES 

NO 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

DAYS  OF  SCHOOL  MISSED:  3-4,  5-10,  10  OR  MORE 

DAYS,  missing 

DAYS  OF  SCHOOL  MISSED:  0-1,  2 DAYS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  5 

NUMBER  OF  INDEPENDENT  CONTRASTS:  4 

ALL  MY  LIFE 

LIVED  IN  US  MORE  THAN  5 YEARS 

LIVED  IN  US  3-5  YEARS 

LIVED  IN  US  LESS  THAN  3 YEARS 

LIVED  IN  US  MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  3 

NUMBER  OF  INDEPENDENT  CONTRASTS:  2 


GRADES  IN  STATE 
GRADES  IN  STATE 
GRADES  IN  STATE 


LESS  THAN  1 GRADE,  MISSING 
1-2  GRADES 
3 OR  MORE  GRADES 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SCHNRM-?  (M  ) 

002  SCHNRM-Y  (®  ) 


SCHLOOOl 

SCHOOL  LEVEL  AVERAGE  MATH  NORMIT  (MISSING  VS  NON-MISSING) 

N04,  S04,  N08,  SOS,  N12 
SCH  NORM 

SCHNORM  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

NUMBER  OF  INDEPENDENT  CONTRASTS: 


SCHOOL  LEVEL  AVERAGE  MATH  NORMIT  MISSING 
SCHOOL  LEVEL  AVERAGE  MATH  NORMIT  NOT-MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0002 

SCHOOL  LEVEL  AVERAGE  MATH  NORMIT 
N04,  S04,  NOS,  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


SNRM-LIN 

SCHNORM 

SCALE 


2 

1 


001  SNRM-LIN  (#  ) (FS.4) 

002  SNRM-LIN  (M  ) 0 


SCHOOL  LEVEL  AVERAGE  MATH  NORMIT  MEAN 
SCHOOL  LEVEL  AVERAGE  MATH  NORMIT  MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK002S 

HOW  MANY  GRADES  IN  THIS  STATE  (12TH  GRADE) 

N12 

STGRAD12 

B00S301  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


4 

3 


001 

STGRD<1 

(1,M 

) 000 

002 

STGRDl-2 

(2 

) 100 

003 

STGRD3-5 

(3 

) 010 

004 

STGRD6> 

(4,5 

) 001 

GRADES  IN  STATE 
GRADES  IN  STATE 
GRADES  IN  STATE 
GRADES  IN  STATE 


LESS  THAN  1 GRADE,  MISSING 

1-2  GRADES 

3-5  GRADES 

MORE  THAN  5 GRADES 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


BACK0029 

HOW  MANY  TIMES  HAVE  YOU  CHANGED  SCHOOLS  IN  PAST  TWO  YEARS  BECAUSE  YOU  MOVED? 


o '78 

ERIC 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES /ASSESSMENTS : 

N04,  S04,  1 

CONDITIONING  VAR  LABEL: 

SCH  CHGS 

NAEP  ID: 

B007301 

TYPE  OF  CONTRAST; 

CLASS 

001  SCHCHG-0  (1  ) 

000 

002  SCHCHG-1  (2  ) 

100 

003  SCHCHG-2  (3  ) 

010 

004  SCHCHG-3  (4,M  ) 

001 

CONDITIONING  VARIABLE  ID: 

BACK0030 

DESCRIPTION: 

HOW  OFTEN  : 

GRADES /ASSESSMENTS : 

N04,  S04,  : 

CONDITIONING  VAR  LABEL: 

DISC0HOM 

NAEP  ID: 

B007401 

TYPE  OF  CONTRAST: 

CLASS 

001  DIS0HOM1  (1  ) 

000 

002  DIS0HOM2  (2  ) 

100 

003  DIS0HOM3  (3  ) 

010 

004  DIS0HOM4  (4,M  ) 

001 

CONDITIONING  VARIABLE  ID: 

BACK0031 

DESCRI PTION: 

ABOUT  HOW 

GRADES/ASSESSMENTS : 

N04,  S04, 

CONDITIONING  VAR  LABEL: 

PGSREADl 

NAEP  ID: 

BOOllOl 

TYPE  OF  CONTRAST: 

CLASS 

001  PGS<6,?  (5,M  ) 

0 

002  PGS>5  (1,2, 3, 4 ) 

1 

CONDITIONING  VARIABLE  ID: 

BACK0032 

DESCRIPTION: 

ABOUT  HOW 

GRADES /ASSESSMENTS : 

N04,  S04, 

CONDITIONING  VAR  LABEL: 

PGSREAD2 

NAEP  ID: 

BOOllOl 

TYPE  OF  CONTRAST: 

CLASS 

001  PGS<11,?  (4,5,M  ) 

0 

002  PGS>10  (1,2,3  ) 

1 1 

CONDITIONING  VARIABLE  ID; 

SUBJOOOl 

DESCRIPTION: 

WHAT  KIND 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

MATTAKE 

TYPE  OF  CONTRAST: 

CLASS 

001  NO  MATH  (01  ! 

1 0000000 

002  8TH  GRD  (02  ! 

1 1000000 

003  PREALG  (03  ! 

1 0100000 

004  ALGEBRA  (04  ! 

1 0010000 

005  INT/SEQ  (05  ! 

1 0001000 

006  APPLIED  (06 

) 0000100 

007  OTHER  (07 

) 0000010 

008  MISSING  (M 

) 0000001 

CONDITIONING  VARIABLE  ID: 

BACK0033 

DESCRIPTION: 

WHICH  BES1 

GRADES/ ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

HS  PROG 

NAEP  ID: 

B008501 

TYPE  OF  CONTRAST: 

CLASS 

001  GENERAL  (1 

) 0000 

002  ACADEMIC  (2 

) 1000 

003  VOC/TECH  (3 

) 0100 

004  OTHERPGM  (4 

) 0010 

005  HS  PGM-?  (M 

) 0001 

CONDITIONING  VARIABLE  ID: 

BACK0034 

DESCRI PTION: 

SEMESTERS 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

SEM  ENG 

NAEP  ID: 

B007101 

TYPE  OF  CONTRAST: 

CLASS 

001  SEMENG-?  (M 

) 0 

002  SEMENG-''  (1-9 

) 1 

CONDITIONING  VARIABLE  ID: 

BACK0035 

DESCRIPTION: 

NUMBER  OF 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

#ENG-LIN 

NAEP  ID: 

B007101 

N08,  SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


SCHOOL  CHANGES 
SCHOOL  CHANGES 
SCHOOL  CHANGES 
SCHOOL  CHANGES 


NONE 

ONE 

TWO 

THREE  OR  MORE, 


MISSING 


N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


DISCUSS  STUDIES  AT  HOME 
DISCUSS  STUDIES  AT  HOME 
DISCUSS  STUDIES  AT  HOME 
DISCUSS  STUDIES  AT  HOME 
MISSING 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MONTH 
NEVER  OR  HARDLY  EVER, 


N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


PAGES  READ: 
PAGES  READ: 


5 OR  FEWER  A DAY,  MISSING 
6-10,  11-15,  16-20,  20  OR  MORE 


SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


PAGES  READ: 
PAGES  READ: 


6-10,  5 OR  FEWER  A DAY,  MISSING 
11-15,  16-20,  20  OR  MORE 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  TAKING  THIS  YEAR 
EIGHTH  GRADE  MATH 
PREALGEBRA 
ALGEBRA 

INTEGRATED/ SEQUENTAL 
APPLIED  MATH 
OTHER  MATH  CLASSUR 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


HIGH  SCHOOL  PROGRAM 
HIGH  SCHOOL  PROGRAM 
HIGH  SCHOOL  PROGRAM 
HIGH  SCHOOL  PROGRAM 
HIGH  SCHOOL  PROGRAM 


GENERAL 

ACADEMIC/COLLEGE  PREP 
VOCATIONAL  OR  TECHNICAL 
OTHER 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


ENGLISH  SEMESTERS; 
ENGLISH  SEMESTERS: 


MISSING 
NOT  MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


530 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST: 

001  #ENG-LIN  (1-9,M=0  ) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  SEMMATOl  (1,2,3  ) 

002  SEMMAT02  (4,5  ) 

003  SEMMAT03  (6,7  ) 

004  SEMMAT04  (8,9  ) 

005  SEMMAT-?  (M  ) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #MAT-LIN  (1-9,M=0  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SEMSCI-?  (M  ) 

002  SEMSCI-^  (1-9  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #SCI-LIN  (1-9, M=0  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SEMHIS-?  (M  ) 

002  SEMHIS-^  (1-9  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #HIS-LIN  (1-9, M=0  ) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SEMFLG-?  (M  ) 

002  SEMFLG-^  (1-9  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #FLG-LIN  (1-9, M=0  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


LINEAR  . NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

0.0  + 1.0*X  NUMBER  OF  SEMESTERS  ENGLISH/ LITERATURE/ WRITING 

(LINEAR) 


BACK0036 

SEMESTERS  MATHEMATICS  (MISSING  VS  NON-MiSSiNG) 
N12 

SEM  MAT 
B007102 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


0000 

1000 

0100 

0010 

0001 

BACK0037 
NUMBER  OF 
N12 

#MAT-LIN 

B007102 

LINEAR 

0.0  + 1. 

BACK0038 

SEMESTERS 

N12 

SEM  SCI 
B007103 
CLASS 

0 

1 

BACK0039 
NUMBER  OF 
N12 

#SCI-LIN 

B007103 

LINEAR 


MATH  SEMEST  ERS : 
MATH  SEMESTERS: 
MATH  SEMESTERS: 
MATH  SEMESTERS: 
MISSING 


ZERO  TO  2 
3 OR  4 
5 OR  6 
7 OR  MORE 


SEMESTERS  MATHEMATICS  (LINEAR) 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

0*X  NUMBER  OF  SEMESTERS  MATHEMATICS  (LINEAR) 

SCIENCE  (MISSING  VS  NON-MISSING) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


SCIENCE  SEMESTERS: 
SCIENCE  SEMESTERS: 


MISSING 
NOT  MISSING 


SEMESTERS  SCIENCE  (LINEAR) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


0.0 


1.0*X 


BACK0040 

SEMESTERS 

N12 

SEM  HIS 
B007104 
CLASS 

0 

1 

BACK0041 
NUMBER  OF 
N12 

#HIS-LIN 

B007104 

LINEAR 


0.0  + 1 .0*X 


NUMBER  OF  SEMESTERS  SCIENCE  (LINEAR) 
HISTORY/SOCIAL  STUDIES/GEOGRAPHY  (MISSING  VS  NON-MiSSiNG) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


H I STORY  SEMESTERS  : MI  S'S  ING 

HISTORY  SEMESTERS:  NOT  MISSING 


SEMESTERS  HISTORY/SOCIAL  STUDIES/GEOGRAPHY  (LINEAR) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NUMBER  OF  SEMESTERS  HISTORY/ SOCIAL 
STUD I ES / GEOGRAPHY  ( LINEAR ) 


BACK0042 

SEMESTERS 

N12 

SEM  FLG 
B007105 
CLASS 

0 

1 

BACK0043 
NUMBER  OF 
N12 

#FLG-LIN 

B007105 

LINEAR 


FOREIGN  LANGUAGES  (MISSING  VS  NON-MISS ING) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

FOREIGN  LANGUAGE ■ SEMESTERS : MISSING 

FOREIGN  LANGUAGE  SEMESTERS:  NOT  MISSING 


SEMESTERS  FOREIGN  LANGUAGES  (LINEAR) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


0.0 


1.0*X 


BACK0044 

SEMESTERS 


NUMBER  OF  SEMESTERS  FOREIGN  LANGUAGES  (LINEAR) 
VOCATIONAL /TECHNICAL /BUS I NESS  EDUCATION  (MISSING  VS  NON-MISSING) 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SEMVOC-?  (M 

002  SEMVOC-''  (1-9 


N12 

SEM  VOC 
B007106 
CLASS 

0 

1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


VOC/TECH  SEMESTERS: 
VOC/TECH  SEMESTERS: 


MISSING 
NOT  MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #VOC-LIN  (1-9,M=0 


BACK0045 

NUMBER  OF  SEMESTERS  VOCATIONAL/TECHNICAL/BUSINESS  EDUCATION  (LINEAR) 
N12 

#VOC-LIN 

B007106  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 0.0 


1.0*X 


NUMBER  OF  SEMESTERS  VOCATIONAL /TECHNICAL /BUSINESS 
EDUCATION  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  SEMART-7 

002  SEMART-^ 


(M 

(1-9 


BACK0046 

SEMESTERS  ART/MUSiC 
N12 

SEM  ART 
B007107 
CLASS 

0 

1 


(MISSING  VS  NON-MISSING) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


ART/MUSIC  SEMESTERS: 
ART/MUSIC  SEMESTERS: 


MISSING 
NOT  MISSING 


CONDITIONING  VARIABLE  ID:  BACK0047 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #ART-LIN  (1-9,M=0 


NUMBER  OF  SEMESTERS  ART/MUSIC  (LINEAR) 
N12 

#ART-LIN 
B007107 
LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 0.0 


1.0*X 


NUMBER  OF  SEMESTERS  ART/MUSIC  (LINEAR) 


CONDITIONING  VARIABLE  ID:  BACK0048 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 


TYPE  OF 

CONTRAST 

001 

G/R 

11 

(11 

002 

G/R 

12 

(12 

003 

G/R 

13 

(13 

004 

G/R 

14 

(14 

005 

G/R 

21 

(21 

006 

G/R 

22 

(22 

007 

G/R 

23 

(23 

OOS 

G/R 

24  • 

(24 

INTERACTION:  GENDER  BY  RACE/ETHNICITY 

N04,  S04,  NOS,  SOS,  N12 

GEND/RAC 

N/A  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

INTERACTION  NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 010101 
) -10000 
) 00-100 
) 0000-1 
) -1-1-1 
) 010000 
) 000100 
) 000001 


GEND/RAC 

GEND/RAC 

GEND/RAC 

GEND/RAC 

GEND/RAC 

GEND/RAC 

GEND/RAC 

GEND/I^C 


INTACT 

INTACT 

INTACT 

INTACT 

INTACT 

INTACT 

INTACT 

INTACT 


1. 

MALE 

1. 

WHI/AI/0 

1. 

MALE 

2 . 

BLACK 

1. 

MALE 

3 . 

HISPANIC 

1. 

MALE 

4 . 

ASIAN 

2 . 

FEMALE 

1. 

WHI/AI/0 

2 . 

FEMALE 

2 . 

BLACK 

2 . 

FEMALE 

3 . 

HISPANIC 

2 . 

FEMALE 

4 . 

ASIAN 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001 

G/T 

11 

(11 

002 

G/T 

12 

(12 

003 

G/T 

13 

(13 

004 

G/T 

14 

(14 

005 

G/T 

15 

(15 

006 

G/T 

21 

(21 

007 

G/T 

22 

(22 

OOS 

G/T 

23 

(23 

009 

G/T 

24 

(24 

010 

G/T 

25 

(25 

BACK0049 

INTERACTION:  GENDER  BY  TYPE  OF  LOCALE  (5  CATEGORIES) 

N04,  S04,  NOS,  SOS,  N12 
GEND/TOL 


N/A 

INTERACTION 

01010101 
-1000000 
00-10000 
0000-100 
) 000000-1 
-l-l-l-l 
01000000 
00010000 
00000100 
00000001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 

GEND/TOL 


10 

4 


INTACT : 

1. 

MALE 

1. 

BIG 

CTY5 

INTACT : 

1. 

MALE 

2 . 

MID 

CTY5 

INTACT : 

1. 

MALE 

3. 

FR/BTWN5 

INTACT : 

1. 

MALE 

4. 

SML 

TWN5 

INTACT : 

1. 

MALE 

5. 

RURAL5 

INTACT : 

2 . 

FEMALE 

1. 

BIG 

CTY5 

INTACT : 

2 . 

FEMALE 

2 . 

MID 

CTY5 

INTACT : 

2 . 

FEMALE 

3 . 

FR/BTWN5 

INTACT : 

2 . 

FEMALE 

4. 

SML 

TWN5 

INTACT : 

2 . 

FEMALE 

5. 

RURAL 5 

CONDITIONING  VARIABLE  ID:  BACK0050 

DESCRIPTION: 

GRADES/ ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 


INTERACTION:  GENDER  BY  PARENTS' 

N04,  S04,  NOS,  SOS,  N12 
GEND/PAR 


NAEP  ID: 

N/A 

TYPE  OF 

CONTRAST : 

INTERACTION 

001 

G/P 

11 

(11 

) 01010101 

002 

G/P 

12 

(12 

) -1000000 

003 

G/P 

13 

(13 

) 00-10000 

004 

G/P 

14 

(14 

) 0000-100 

005 

G/P 

15 

(15 

) 000000-1 

006 

G/P 

21 

(21 

) -l-l-l-l 

007 

G/P 

22 

(22 

) 01000000 

OOS 

G/P 

23 

(23 

) 00010000 

009 

G/P 

24 

(24 

) 00000100 

EDUCATION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


GEND/PAR 

GEND/PAR 

GEND/PAR 

GEND/PAR 

GEND/PAR 

GEND/PAR 

GEND/PAR 

GEND/PAR 

GEND/PAR 


10 

4 


INTACT : 

1. 

MALE 

1. 

< HS 

INTACT : 

1. 

MALE 

2. 

HS  GRAD 

INTACT : 

1. 

MALE 

3 . 

POST  HS 

INTACT : 

1. 

MALE 

4 . 

COL  GRAD 

INTACT : 

1. 

MALE 

5. 

PARED-? 

INTACT : 

2. 

FEMALE 

1. 

< HS 

INTACT : 

2. 

FEMALE 

2 . 

HS  GRAD 

INTACT : 

2. 

FEMALE 

3. 

POST  HS 

INTACT : 

2. 

FEMALE 

4 . 

COL  GRAD 

ERIC 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


010  G/P  25 


(25 


) 00000001 


GEND/ PAR  INTACT : 2 . FEMALE 


CONDITIONING  VARIABLE  ID:  BACK0051 


DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  L7VBEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


INTERACTION:  GENDER  BY  SCHOOL  TYPE 

N04,  S04,  NOS,  SOS,  N12 

GEND/SCH 


N/A 

INTERACTION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  G/S  11 

002  G/S  12 

003  G/S  13 

004  G/S  21 

005  G/S  22 

006  G/S  23 


(11 

(12 

(13 

(21 

(22 

(23 


0101 

GEND/SCH 

INTACT : 

1. 

MALE 

1. 

PUBLIC 

-100 

GEND/SCH 

INTACT : 

1. 

MALE 

2 . 

PRIVATE 

00-1 

GEND/SCH 

INTACT : 

1. 

MALE 

3 . 

CATHOLIC 

-1-1 

GEND/SCH 

INTACT : 

2 . 

FEMALE 

1. 

PUBLIC 

0100 

GEND/SCH 

INTACT : 

2 . 

FEMALE 

2 . 

PRIVATE 

0001 

GEND/SCH 

INTACT : 

2 . 

FEMALE 

3. 

CATHOLIC 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


NAEP  ID; 

N/A 

TYPE  OF 

CONTRAST : 

INTERACTION 

001 

R/T 

11 

(11 

).  010101010101010101010101 

002 

R/T 

12 

(12 

) -1000000-1000000-1000000 

003 

R/T 

13 

(13 

) 00-1000000-1000000-10000 

004 

R/T 

14 

(14 

) 0000-1000000-1000000-100 

005 

R/T 

15 

(15 

) 000000-1000000-1000000-1 

006 

R/T 

21 

(21 

) -1-1-1-10000000000000000 

007 

R/T 

22 

(22 

) 010000000000000000000000 

OOS 

R/T 

23 

(23 

) 000100000000000000000000 

009 

R/T 

24 

(24 

) 000001000000000000000000 

010 

R/T 

25 

(25 

) 000000010000000000000000 

Oil 

R/T 

31 

(31 

) 00000000-1-1-1-100000000 

012 

R/T 

32 

(32 

) 000000000100000000000000 

013 

R/T 

33 

(33 

) 000000000001000000000000 

014 

R/T 

34 

(34 

) 000000000000010000000000 

015 

R/T 

35 

(35 

) 000000000000000100000000 

016 

R/T 

41 

(41 

) 0000000000000000-1-1-1-1 

017 

R/T 

42 

(42 

) 000000000000000001000000 

OIS 

R/T 

43 

(43 

) 000000000000000000010000 

019 

R/T 

44 

(44 

) 000000000000000000000100 

020 

R/T 

45 

(45 

) 000000000000000000000001 

BACK0052 

INTERACTION:  RACE/ETHNICITY  BY  TYPE  OF  LOCALE  (5  CATEGORIES) 

N04,  S04,  NOS,  SOS,  N12 
RACE/TOL 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 
RACE/TOL 


20 

12 


INTACT : 

1 . 

WHl/Al/0 

1. 

BIG  CTY5 

INTACT : 

1. 

WHI/AI/0 

2 . 

MID  CTY5 

INTACT : 

1. 

WHI/AI/0 

3 . 

FR/BTWN5 

INTACT : 

1. 

WHI/AI/0 

4. 

SML  TWN5 

INTACT : 

1 . 

WHI/AI/0 

5. 

RURAL5 

INTACT : 

2 . 

BLACK 

1. 

BIG  CTY5 

INTACT : 

2 . 

BLACK 

2 . 

MID  CTY5 

INTACT : 

2 . 

BLACK 

3 . 

FR/BTWN5 

INTACT : 

2 . 

BLACK 

4 . 

SML  TWN5 

INTACT : 

2 . 

BLACK 

5. 

RURAL5 

INTACT : 

3 . 

HISPANIC 

1. 

BIG  CTY5 

INTACT : 

3 . 

HISPANIC 

2 . 

MID  CTY5 

INTACT: 

3 . 

HISPANIC 

3 . 

FR/BTWN5 

INTACT : 

3 . 

HISPANIC 

4 . 

SML  TWN5 

INTACT : 

3 . 

HISPANIC 

5. 

RURAL5 

INTACT: 

4. 

ASIAN 

1 . 

BIG  CTY5 

INTACT: 

4. 

ASIAN 

2 . 

MID  CTY5 

INTACT : 

4. 

ASIAN 

3 . 

FR/BTWN5 

INTACT : 

4 . 

ASIAN 

4 . 

SML  TWN5 

INTACT: 

4. 

ASIAN 

5. 

RURAL5 

CONDITIONING  VARIABLE  ID:  BACK0053 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TYPE  OF 

CONTRAST 

001 

R/P 

11 

(11 

002 

R/P 

12 

(12 

003 

R/P 

13 

(13 

004 

R/P 

14 

(14 

005 

R/P 

15 

(15 

006 

R/P 

21 

(21 

007 

R/P 

22 

(22 

008 

R/P 

23 

(23 

009 

R/P 

24 

(24 

010 

R/P 

25 

(25 

Oil 

R/P 

31 

(31 

012 

R/P 

32 

(32 

013 

R/P 

33 

(33 

014 

R/P 

34 

(34 

015 

R/P 

35 

(35 

016 

R/P 

41 

(41 

017 

R/P 

42 

(42 

018 

R/P 

43 

(43 

019 

R/P 

44 

(44 

020 

R/P 

45 

(45 

INTERACTION:  RACE/ETHNICITY  BY  PARENTS 

N04,  S04,  NOS,  SOS,  N12 
RACE /PAR 
N/A 

INTERACTION 


EDUCATION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


20 

12 


010101010101010101010101 

-1000000-1000000-1000000 

00-1000000-1000000-10000 

0000-1000000-1000000-100 

000000-1000000-1000000-1 

-1-1-1-10000000000000000 

010000000000000000000000 

000100000000000000000000 

000001000000000000000000 

000000010000000000000000 

00000000-1-1-1-100000000 

000000000100000000000000 

000000000001000000000000 

000000000000010000000000 

000000000000000100000000 

0000000000000000-1-1-1-1 

000000000000000001000000 

000000000000000000010000 

000000000000000000000100 

000000000000000000000001 


RACE /PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE /PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE /PAR 
RACE /PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE /PAR 
RACE /PAR 


INTACT : 

1. 

WHI/AI/0 

1. 

< HS 

INTACT : 

1. 

WHI/AI/0 

2 . 

HS  GRAD 

INTACT : 

1. 

WHI/AI/0 

3 . 

POST  HS 

INTACT : 

1. 

WHI/AI/0 

4. 

COL  GRAD 

INTACT : 

1. 

WHI/AI/0 

5. 

PARED-? 

INTACT : 

2 . 

BLACK 

1 . 

< HS 

INTACT: 

2. 

BLACK 

2 . 

HS  GRAD 

INTACT: 

2 . 

BLACK 

3 . 

POST  HS 

INTACT : 

2 . 

BLACK 

4 . 

COL  GRAD 

INTACT : 

2 . 

BLACK 

5. 

PARED-? 

INTACT: 

3 . 

HISPANIC 

1 . 

< HS 

INTACT: 

3 . 

HISPANIC 

2 . 

HS  GRAD 

INTACT: 

3 . 

HISPANIC 

3 . 

POST  HS 

INTACT : 

3 . 

HISPANIC 

4 . 

COL  GRAD 

INTACT : 

3 . 

HISPANIC 

5. 

PARED-? 

INTACT : 

4. 

ASIAN 

1 . 

< HS 

INTACT : 

4. 

ASIAN 

2 . 

HS  GRAD 

INTACT : 

4. 

ASIAN 

3 . 

POST  HS 

INTACT : 

4. 

ASIAN 

4 . 

COL  GRAD 

INTACT : 

4. 

ASIAN 

5. 

PARED-? 

CONDITIONING  VARIABLE  ID:  BACK0054 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


INTERACTION:  RACE/ETHNICITY  BY  SCHOOL  TYPE 

N04,  S04,  NOS,  SOS,  N12 

RACE/SCH 


NAEP  ID! 

N/A 

TYPE  OF 

CONTRAST : 

INTERACTION 

001 

R/S 

11 

(11 

) 010101010101 

002 

R/S 

12 

(12 

) -100-100-100 

003 

R/S 

13 

(13 

) 00-100-100-1 

004 

R/S 

21 

(21 

) -1-100000000 

005 

R/S 

22 

(22 

) 010000000000 

006 

R/S 

23 

(23 

) 000100000000 

007 

R/S 

31 

(31 

) 0000-1-10000 

008 

R/S 

32 

(32 

) 000001000000 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 


12 

6 


INTACT : 

1 . 

WHI/AI/0 

1 . 

PUBLIC 

INTACT : 

1. 

WHI/AI/0 

2 . 

PRIVATE 

INTACT : 

1. 

WHI/AI/0 

3. 

CATHOLIC 

INTACT : 

2. 

BLACK 

1. 

PUBLIC 

INTACT : 

2. 

BLACK 

2. 

PRIVATE 

INTACT : 

2. 

BLACK 

3. 

CATHOLIC 

INTACT : 

3. 

HISPANIC 

1. 

PUBLIC 

INTACT : 

3. 

HISPANIC 

2. 

PRIVATE 

Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


009 

R/S 

33 

(33 

) 000000010000 

RACE/SCH 

INTACT : 

3 . 

HISPANIC 

3. 

CATHOLIC 

010 

R/S 

41 

(41 

) 00000000-1-1 

RACE/SCH 

INTACT : 

4 . 

ASIAN 

1. 

PUBLIC 

Oil 

R/S 

42 

(42 

) 000000000100 

RACE/SCH 

INTACT : 

4 . 

ASIAN 

2. 

PRIVATE 

012 

R/S 

43 

(43 

) 000000000001 

RACE/SCH 

INTACT : 

4. 

ASIAN 

3. 

CATHOLIC 

CONDITIONING  VARIABLE  IE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TYPE  OF 

CONTRAST 

001 

T/P 

11 

(11 

002 

T/P 

12 

(12 

003 

T/P 

13 

(13 

004 

T/P 

14 

(14 

005 

T/P 

15 

(15 

006 

T/P 

21 

(21 

007 

T/P 

22 

(22 

008 

T/P 

23 

(23 

009 

T/P 

24 

(24 

010 

T/P 

25 

(25 

Oil 

T/P 

31 

(31 

012 

T/P 

32 

(32 

013 

T/P 

33 

(33 

014 

T/P 

34  ■ 

(34 

015 

T/P 

35 

(35 

016 

T/P 

41 

(41 

017 

T/P 

42 

(42 

018 

T/P 

43 

(43 

019 

T/P 

44 

(44 

020 

T/P 

45 

(45 

021 

T/P 

51 

(51 

022 

T/P 

52 

(52 

023 

T/P 

53 

(53 

024 

T/P 

54 

(54 

025 

T/P 

55 

(55 

BACK0055 
INTERACTION: 
N04,  S04,  NOS, 
TOL5/PAR 
N/A 

INTERACTION 


TYPE  OF  LOCALE 
SOS,  N12 


(5  CATEGORIES)  BY  PARENT'S  EDUCATION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


25 

16 


01010101010101010101010101010101  TOL5/PAR 
-1000000-1000000-1000000-1000000  TOL5/PAR 
00-1000000-1000000-1000000-10000  TOL5/PAR 
0000-1000000-1000000-1000000-100  TOL5/PAR 
000000-1000000-1000000-1000000-1  TOL5/PAR 
-1-1-1-1000000000000000000000000  TOL5/PAR 
01000000000000000000000000000000  TOL5/PAR 
00010000000000000000000000000000  TOL5/PAR 
00000100000000000000000000000000  TOL5/PAR 
00000001000000000000000000000000  TOL5/PAR 
00000000-1-1-1-10000000000000000  TOL5/PAR 
00000000010000000000000000000000  TOL5/PAR 
00000000000100000000000000000000  TOL5/PAR 
00000000000001000000000000000000  TOL5/PAR 
00000000000000010000000000000000  TOL5/PAR 
0000000000000000-1-1-1-100000000  TOL5/PAR 
00000000000000000100000000000000  TOL5/PAR 
00000000000000000001000000000000  TOL5/PAR 
00000000000000000000010000000000  TOL5/PAR 
00000000000000000000000100000000  TOL5/PAR 
000000000000000000000000-1-1-1-1  TOL5/PAR 
00000000000000000000000001000000  TOL5/PAR 
00000000000000000000000000010000  TOL5/PAR 
00000000000000000000000000000100  TOL5/PAR 
00000000000000000000000000000001  TOL5/PAR 


INTACT : 

1. 

BIG 

CTY5 

1. 

< HS 

INTACT : 

1. 

BIG 

CTY5 

2 . 

HS  GRAD 

INTACT : 

1. 

BIG 

CTY5 

3. 

POST  HS 

INTACT : 

1. 

BIG 

CTY5 

4 . 

COL  GRAD 

INTACT : 

1. 

BIG 

CTY5 

5. 

PARED-? 

INTACT : 

2 . 

MID 

CTY5 

1. 

< HS 

INTACT : 

2 . 

MID 

CTY5 

2 . 

HS  GRAD 

INTACT : 

2 . 

MID 

CTY5 

3 . 

POST  HS 

INTACT : 

2 . 

MID 

CTY5 

4. 

COL  GRAD 

INTACT : 

2 . 

MID 

CTY5 

5. 

PARED-? 

INTACT : 

3 . 

FR/BTWN5 

1. 

< HS 

INTACT : 

3 . 

FR/B'TWN5 

2 . 

HS  GRAD 

INTACT : 

3 . 

FR/BTWN5 

3 . 

POST  HS 

INTACT : 

3 . 

FR/BTWN5 

4. 

COL  GRAD 

INTACT : 

3 . 

FR/BTWN5 

5. 

PARED-? 

INTACT : 

4. 

SML 

TWN5 

1. 

< HS 

INTACT : 

4. 

SML 

'TWN5 

2 . 

HS  GRAD 

INTACT : 

4. 

SML 

TWN5 

3 . 

POST  HS 

INTACT: 

4. 

SML 

rms 

4,. 

COL  GRAD 

INTACT : 

4. 

SML 

TWN5 

5 . 

PARED-? 

INTACT: 

5. 

RURAL 5 

1. 

< HS 

INTACT: 

5. 

RURAL 5 

2 . 

HS  GRAD 

INTACT : 

5. 

RURAL 5 

3 . 

POST  HS 

INTACT : 

5. 

RURAL 5 

4 . 

COL  GRAD 

INTACT : 

5. 

RURAL 5 

5 . 

PARED-? 

CONDITIONING  VARIABLE  ID:  BACK0056 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


NAEP  ID: 

TYPE  OF 

CONTRAST 

001 

T/S 

11 

(11 

002 

T/S 

12 

(12 

003 

T/S 

13 

(13 

004 

T/S 

21 

(21 

005 

T/S 

22 

(22 

006 

T/S 

23 

(23 

007 

T/S 

31 

(31 

008 

T/S 

32 

(32 

009 

T/S 

33 

(33 

010 

T/S 

41 

(41 

Oil 

T/S 

42 

(42 

012 

T/S 

43 

(43 

013 

T/S 

51 

(51 

014 

T/S 

52 

(52 

015 

T/S 

53 

(53 

INTERACTION: 
N04,  S04,  NOS 
TOL5/SCH 
N/A 

INTERACTION 


TYPE  OF  LOCALE  (5  CATEGORIES)  BY  SCHOOL  TYPE 
SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


0101010101010101 

-100-100-100-100 

00-100-100-100-1 

-1-1000000000000 

0100000000000000 

0001000000000000 

0000-1-100000000 

0000010000000000 

0000000100000000 

00000000-1-10000 

0000000001000000 

0000000000010000 

000000000000-1-1 

0000000000000100 

0000000000000001 


15 

s 


TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 


INTACT : 

1. 

BIG 

CTY5 

1. 

PUBLIC 

INTACT: 

1. 

BIG 

CTY5 

2 . 

PRIVATE 

INTACT: 

1. 

BIG 

CTY5 

3 . 

CATHOLIC 

INTACT: 

2 . 

MID 

CTY5 

1 . 

PUBLIC 

INTACT: 

2 . 

MID 

CTY5 

2 . 

PRIVATE 

INTACT: 

2 . 

MID 

CTY5 

3 . 

CATHOLIC 

INTACT: 

3 . 

FR/B'TWN5 

1. 

PUBLIC 

INTACT: 

3 . 

FR/BTWN5 

2 . 

PRIVATE 

INTACT : 

3 . 

FR/BTWN5 

3 . 

CATHOLIC 

INTACT : 

4 . 

SML 

TWN5 

1. 

PUBLIC 

INTACT : 

4 . 

SML 

TWN5 

2 . 

PRIVATE 

INTACT : 

4 . 

SML 

TWN5 

3 . 

CATHOLIC 

INTACT : 

5. 

RURAL 5 

1. 

PUBLIC 

INTACT : 

5. 

RURAL 5 

2 . 

PRIVATE 

INTACT : 

5. 

RURAL5 

3 . 

CATHOLIC 

CONDITIONING  VARIABLE  ID:  BACK0057 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


NAEP 

ID: 

TYPE 

OF 

CONTRAST 

001 

P/S 

11 

(11 

002 

P/S 

12 

(12 

003 

P/S 

13 

(13 

004 

P/S 

21 

(21 

005 

P/S 

22 

(22 

006 

P/S 

23 

(23 

007 

P/S 

31 

(31 

008 

P/S 

32 

(32 

009 

P/S 

33 

(33 

010 

P/S 

41 

(41 

Oil 

P/S 

42 

(42 

012 

P/S 

43 

(43 

013 

P/S 

51 

(51 

014 

P/S 

52 

(52 

015 

P/S 

53 

(53 

INTERACTION: 
N04,  S04,  NOS 
PARE/SCH 
N/A 

INTERACTION 


0101010101010101 

-100-100-100-100 

00-100-100-100-1 

-1-1000000000000 

0100000000000000 

0001000000000000 

0000-1-100000000 

0000010000000000 

0000000100000000 

00000000-1-10000 

0000000001000000 

0000000000010000 

000000000000-1-1 

0000000000000100 

0000000000000001 


PARENTS'  EDUCATION  BY  SCHOOL  TYPE 
, SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 

PARE/SCH 


15 

S 


INTACT : 

1. 

< HS 

1. 

PUBLIC 

INTACT: 

1. 

< HS 

2. 

PRIVATE 

INTACT : 

1 . 

< HS 

3 . 

CATHOLIC 

INTACT : 

2 . 

HS  GRAD 

1. 

PUBLIC 

INTACT : 

2 . 

HS  GRAD 

2 . 

PRIVATE 

INTACT : 

2 . 

HS  GRAD 

3 . 

CATHOLIC 

INTACT : 

3 . 

POST  HS 

1. 

PUBLIC 

INTACT : 

3 . 

POST  HS 

2 . 

PRIVATE 

INTACT : 

3 . 

POST  HS 

3 . 

CATHOLIC 

INTACT : 

4. 

COL  GRAD 

1. 

PUBLIC 

INTACT : 

4. 

COL  GRAD 

2. 

PRIVATE 

INTACT : 

4. 

COL  GRAD 

3 . 

CATHOLIC 

INTACT : 

5. 

PARED- ? 

1. 

PUBLIC 

INTACT : 

5. 

PARED- ? 

2 . 

PRIVATE 

INTACT : 

5. 

PARED-? 

3. 

CATHOLIC 

CONDITIONING  VARIABLE  ID:  BACK005S 

DESCRIPTION:  INTERACTION: 

GRADES /ASSESSMENTS:  N08,  SOS 


GENDER  BY  MATH  COURSES  TAKING  THIS  YEAR 


ERIC 


BECT  COPY  AVAIWiy 


534 


483 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001 

G/ 

11 

(11 

002 

G/ 

12 

(12 

003 

G/ 

13 

(13 

004 

G/ 

14 

(14 

005 

G/ 

15 

(15 

006 

G/ 

16 

(16 

007 

G/ 

17 

(17 

OOS 

G/ 

IS 

(IS 

009 

G/ 

21 

(21 

010 

G/ 

22 

(22 

Oil 

G/ 

23 

(23 

012 

G/ 

24 

(24 

013 

G/ 

25 

(25 

014 

G/ 

26 

(26 

015 

G/ 

27 

(27 

016 

G/ 

2S 

(2S 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


GEND/ 

N/A 

INTERACTION 

01010101010101 

-1000000000000 

00-10000000000 

0000-100000000 

000000-1000000 

00000000-10000 

0000000000-100 

000000000000-1 

-l-l-l-l-l-l-l 

01000000000000 

00010000000000 

00000100000000 

00000001000000 

00000000010000 

00000000000100 

00000000000001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 


INTACT : 

1. 

MALE 

1. 

NO  MATH 

INTACT : 

1. 

MALE 

2 . 

STH  GRD 

INTACT : 

1. 

MALE 

3. 

PREALG 

INTACT : 

1. 

MALE 

4. 

ALGEBRA 

INTACT : 

1. 

MALE 

5. 

INT/SEQ 

INTACT : 

1. 

MALE 

6. 

APPLIED 

INTACT : 

1. 

MALE 

7 . 

OTHER 

INTACT : 

1. 

MALE 

S. 

MISSING 

INTACT : 

2 . 

FEMALE 

1. 

NO  MATH 

INTACT : 

2 . 

FEMALE 

2 . 

STH  GRD 

INTACT : 

2. 

FEMALE  . 

3. 

PREALG 

INTACT : 

2 . 

FEMALE 

4 . 

ALGEBRA 

INTACT : 

2 . 

FEMALE 

5. 

INT/SEQ 

INTACT : 

2 . 

FEMALE 

6. 

APPLIED 

INTACT : 

2 . 

FEMALE 

7 . 

OTHER 

INTACT : 

2 . 

FEMALE 

S. 

MISSING 

BACK0060 
INTERACTION: 
NOS,  SOS 
RACE/ 


RACE/ETHNICITY  BY  MATH  COURSES  TAKING  THIS  YEAR 


NAEP  ID 

N/A 

TOTAL  NUMBER  OF 

SPECIFIED  CONTRASTS: 

TYPE  OF 

CONTRAST : 

INTERACTION 

NUMBER 

OF  INDEPENDENT  CONTRASTS 

001 

R/ 

11 

(11 

) 333333333333333333333 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

1. 

NO  MATH 

002 

R/ 

12 

(12 

) 122222212222221222222 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

2 . 

STH  GRD 

003 

R/ 

13 

(13 

) 212222221222222122222 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

3 . 

PREALG 

004 

R/ 

14 

(14 

) 221222222122222212222 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

4. 

ALGEBRA 

005 

R/ 

15 

(15 

) 222122222212222221222 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

5. 

INT/SEQ 

006 

R/ 

16 

(16 

) 222212222221222222122 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

6. 

APPLIED 

007 

R/ 

17 

(17 

) 222221222222122222212 

RACE/ 

INTACT ; 

: 1. 

WHI/AI/0 

7 . 

OTHER 

OOS 

R/ 

IS 

(IS 

) 222222122222212222221 

RACE/ 

INTACT 

: 1. 

WHI/AI/0 

S. 

MISSING 

009 

R/ 

21 

(21 

} 111111122222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

1. 

NO  MATH 

010 

R/ 

22 

(22 

) 322222222222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

2 . 

STH  GRD 

Oil 

R/ 

23 

(23 

) 232222222222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

3 . 

PREALG 

012 

R/ 

24 

(24 

} 223222222222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

4. 

ALGEBRA 

013 

R/ 

25 

(25 

) 222322222222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

5. 

INT/SEQ 

014 

R/ 

26 

(26 

) 222232222222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

6. 

APPLIED 

015 

R/ 

27 

(27 

) 222223222222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

7 . 

OTHER 

016 

R/ 

2S 

(2S 

) 222222322222222222222 

RACE/ 

INTACT ; 

: 2. 

BLACK 

S. 

MISSING 

017 

R/ 

31 

(31 

) 222222211111112222222 

RACE/ 

INTACT : 

: 3. 

HISPANIC 

1. 

NO  MATH 

OIS 

R/ 

32 

(32 

) 222222232222222222222 

RACE/ 

INTACT : 

: 3. 

HISPANIC 

2 . 

STH  GRD 

019 

R/ 

33 

(33 

) 222222223222222222222 

RACE/ 

INTACT : 

: 3. 

HISPANIC 

3 . 

PREALG 

020 

R/ 

34 

(34 

} 222222222322222222222 

RACE/ 

INTACT : 

: 3. 

HISPANIC 

4 . 

ALGEBRA 

021 

R/ 

35 

(35 

} 222222222232222222222 

RACE/ 

INTACT : 

: 3. 

HISPANIC 

5. 

INT/SEQ 

022 

R/ 

36 

(36 

) 222222222223222222222 

RACE/ 

INTACT : 

3. 

HISPANIC 

6. 

APPLIED 

023 

R/ 

37 

(37 

) 222222222222322222222 

RACE/ 

INTACT : 

3. 

HISPANIC 

7 . 

OTHER 

024 

R/ 

3S 

(3S 

} 222222222222232222222 

RACE/ 

INTACT : 

3. 

HISPANIC 

S. 

MISSING 

025 

R/ 

41 

(41 

) 222222222222221111111 

RACE/ 

INTACT : 

4. 

ASIAN 

1. 

NO  MATH 

026 

R/ 

42 

(42 

} 222222222222223222222 

RACE/ 

INTACT : 

4. 

ASIAN 

2. 

STH  GRD 

027 

R/ 

43 

(43 

) 222222222222222322222 

RACE/ 

INTACT : 

4. 

ASIAN 

3. 

PREALG 

02S 

R/ 

44 

(44 

) 222222222222222232222 

RACE/ 

INTACT : 

4. 

ASIAN 

4. 

ALGEBRA 

029 

R/ 

45 

(45 

) 222222222222222223222 

RACE/ 

INTACT : 

4 . 

ASIAN 

5. 

INT/SEQ 

030 

R/ 

46 

(46 

) 222222222222222222322 

RACE/ 

INTACT : 

4. 

ASIAN 

6. 

APPLIED 

031 

R/ 

47 

(47 

) 222222222222222222232 

RACE/ 

INTACT : 

4. 

ASIAN 

7. 

OTHER 

032 

R/ 

4S 

(4S 

) 222222222222222223223 

RACE/ 

INTACT : 

4. 

ASIAN 

S. 

MISSING 

CONDITIONING 

VARIABLE 

ID:  BACK0062 

DESCRIPTION: 

INTERACTION : PARENTS ' 

EDUCATION  BY  MATH 

COURSES 

TAKING  THIS  YEAR 

GRADES /ASSESSMENTS : 

NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TYPE  OF 

CONTRAST 

001 

P/ 

11 

(11 

002 

P/ 

12 

(12 

003 

P/ 

13 

(13 

004 

P/ 

14 

(14 

005 

P/ 

15 

(15 

006 

P/ 

16 

(16 

007 

P/ 

17 

(17 

OOS 

P/ 

IS 

(18 

009 

P/ 

21 

(21 

010 

P/ 

22 

(22 

Oil 

P/ 

23 

(23 

012 

P/ 

24 

(24 

013 

P/ 

25 

(25 

014 

P/ 

26 

(26 

015 

P/ 

27 

(27 

016 

P/ 

2S 

(2S 

017 

P/ 

31 

(31 

OIS 

P/ 

32 

(32 

019 

P/ 

33 

(33 

PARE/ 

N/A 

INTERACTION 

) 3333333333333333333333333333 
) 1222222122222212222221222222 
) 2122222212222221222222122222 
) 2212222221222222122222212222 
) 2221222222122222212222221222 
) 2222122222212222221222222122 
) 2222212222221222222122222212 
) 2222221222222122222212222221 
) 1111111222222222222222222222 
) 3222222222222222222222222222 
) 2322222222222222222222222222 
) 2232222222222222222222222222 
) 2223222222222222222222222222 
) 2222322222222222222222222222 
) 2222232222222222222222222222 
) 2222223222222222222222222222 
) 2222222111111122222222222222 
) 2222222322222222222222222222 
) 2222222232222222222222222222 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 

PARE/ 


INTACT : 

1. 

< HS 

1. 

NO  MATH 

INTACT : 

1. 

< HS 

2 . 

STH  GRD 

INTACT : 

1. 

< HS 

3. 

PREALG 

INTACT : 

1. 

< HS 

4. 

ALGEBRA 

INTACT : 

1. 

< HS 

5. 

INT/SEQ 

INTACT : 

1. 

< HS 

6. 

APPLIED 

INTACT : 

1. 

< HS 

7. 

OTHER 

INTACT : 

1 . 

< HS 

S. 

MISSING 

INTACT : 

2 . 

HS  GRAD 

1. 

NO  MATH 

INTACT : 

2 . 

HS  GRAD 

2 . 

STH  GRD 

INTACT : 

2 . 

HS  GRAD 

3. 

PREALG 

INTACT : 

2 . 

HS  GRAD 

4. 

ALGEBRA 

INTACT : 

2 . 

HS  GRAD 

5. 

INT/SEQ 

INTACT : 

2 . 

HS  GRAD 

6. 

APPLIED 

INTACT : 

2 . 

HS  GRAD 

7 . 

OTHER 

INTACT : 

2. 

HS  GRAD 

S. 

MISSING 

INTACT : 

3. 

POST  HS 

1. 

NO  MATH 

INTACT : 

3. 

POST  HS 

2. 

STH  GRD 

INTACT : 

3. 

POST  HS 

3. 

PREALG 

O "84 

ERIC 
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020 

P/ 

34 

(34 

) 2222222223222222222222222222 

PARE/ 

INTACT 

3. 

POST  HS 

4 . 

ALGEBRA 

021 

P/ 

35 

(35 

) 2222222222322222222222222222 

PARE/ 

INTACT 

3. 

POST  HS 

5. 

INT/SEQ 

022 

P/ 

36 

(36 

) 2222222222232222222222222222 

PARE/ 

INTACT 

3. 

POST  HS 

6. 

APPLIED 

023 

P/ 

37 

(37 

) 2222222222223222222222222222 

PARE/ 

INTACT 

3. 

POST  HS 

7 . 

OTHER 

02  4 

P/ 

3S 

(3S 

) 2222222222222322222222222222 

PARE/ 

INTACT 

3. 

POST  HS 

S. 

MISSING 

025 

P/ 

41 

(41 

) 2222222222222211111112222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

1. 

NO  MATH 

026 

P/ 

42 

(42 

. ) 2222222222222232222222222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

2. 

STH  GRD 

027 

P/ 

43 

(43 

) 2222222222222223222222222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

3. 

PREALG 

02S 

P/ 

44 

(44 

) 2222222222222222322222222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

4. 

ALGEBRA 

029 

P/ 

45 

(45 

) 2222222222222222232222222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

5. 

INT/SEQ 

030 

P/ 

46 

(46 

) 2222222222222222223222222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

6. 

APPLIED 

031 

P/ 

47 

(47 

) 2222222222222222222322222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

7 . 

OTHER ■ 

032 

P/ 

4S 

(4S 

) 2222222222222222222232222222 

PARE/ 

INTACT 

4. 

COL  GRAD 

S. 

MISSING 

033 

P/ 

51 

(51 

) 2222222222222222222221111111 

PARE/ 

INTACT 

5. 

PARED-? 

1. 

NO  MATH 

034 

P/ 

52 

(52 

) 2222222222222222222223222222 

PARE/ 

INTACT 

5. 

PARED-? 

2 . 

STH  GRD 

035 

P/ 

53 

(53 

} 2222222222222222222222322222 

PARE/ 

INTACT 

5. 

PARED-? 

3. 

PREALG 

036 

P/ 

54 

{54 

) 2222222222222222222222232222 

PARE/ 

INTACT 

5. 

PARED-? 

4 . 

ALGEBRA 

037 

P/ 

55 

(55 

) 2222222222222222222222223222 

PARE/ 

INTACT 

5. 

PARED-? 

5 . 

INT/SEQ 

03S 

P/ 

56 

(56 

) 2222222222222222222222222322 

PARE/ 

INTACT 

5. 

PARED-? 

6. 

APPLIED 

039 

P/ 

57 

{57 

) 2222222222222222222222222232 

PARE/ 

INTACT 

5. 

PARED-? 

7 . 

OTHER 

040 

P/ 

5S 

(5S 

) 2222222222222222222222222223 

PARE/ 

INTACT 

5. 

PARED-? 

S. 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

grades/assessments : 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0064 
INTERACTION: 
NOS,  SOS 
TOL5/ 

N/A 

INTERACTION 


TYPE  OF  LOCALE  (5  CATEGORIES)  BY  MATH  COURSES  TAKING  THIS  YEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


40 

2S 


001 

T/ 

11 

(11  ) 

3333333333333333333333333333 

TOL5/ 

INTACT: 

1. 

BIG  CTY5 

1 . 

NO  MATH 

002 

T/ 

12 

(12  ) 

1222222122222212222221222222 

TOL5/ 

INTACT: 

1. 

BIG  CTY5 

2 . 

STH  GRD 

003 

T/ 

13 

(13  ) 

2122222212222221222222122222 

TOL5/ 

INTACT: 

1. 

BIG  CTY5 

3 . 

PREALG 

004 

T/ 

14 

(14  ) 

2212222221222222122222212222 

TOL5/ 

INTACT : 

1. 

BIG  CTY5 

4 . 

ALGEBRA 

005 

T/ 

15 

(15  ) 

2221222222122222212222221222 

TOL5/ 

INTACT : 

1. 

BIG  CTY5 

5 . 

INT/SEQ 

006 

T/ 

16 

(16  ) 

2222122222212222221222222122 

TOL5/ 

INTACT: 

1. 

BIG  CTY5 

6. 

APPLIED 

007 

T/ 

17 

(17  ) 

2222212222221222222122222212 

TOL5/ 

INTACT: 

1. 

BIG  CTY5 

7 . 

OTHER 

OOS 

T/ 

IS 

(IS  ) 

2222221222222122222212222221 

TOL5/ 

INTACT: 

1. 

BIG  CTY5 

S. 

MISSING 

009 

T/ 

21 

(21  ) 

1111111222222222222222222222 

TOL5/ 

INTACT : 

2 . 

MID  CTY5 

1 . 

NO  MATH 

010 

T/ 

22 

(22  ) 

3222222222222222222222222222 

TOL5/ 

INTACT: 

2 . 

MID  CTY5 

2 . 

STH  GRD 

Oil 

T/ 

23 

(23  ) 

2322222222222222222222222222 

TOL5/ 

INTACT : 

2 . 

MID  CTY5 

3 . 

PREALG 

012 

T/ 

24 

(24  ) 

2232222222222222222222222222 

TOL5/ 

INTACT : 

2 , 

MID  CTY5 

4. 

ALGEBRA 

013 

T/ 

25 

(25  ) 

2223222222222222222222222222 

TOL5/ 

INTACT : 

2 , 

MID  CTY5 

5. 

INT/SEQ 

014 

T/ 

26 

(26  ) 

2222322222222222222222222222 

TOL5/ 

INTACT : 

2 . 

MID  CTY5 

6. 

APPLIED 

015 

T/ 

27 

(27  > 

2222232222222222222222222222 

TOL5/ 

INTACT : 

2 . 

MID  CTY5 

7. 

OTHER 

016 

T/ 

2S 

(2S’  ) 

2222223222222222222222222222 

TOL5/ 

INTACT : 

2 . 

MID  CTY5 

S. 

MISSING 

017 

T/ 

31 

(31  ) 

2222222111111122222222222222 

TOL5/ 

INTACT: 

3 . 

FR/BTWN5 

1, 

NO  MATH 

OIS 

T/ 

32 

(32  ) 

2222222322222222222222222222 

TOL5/ 

INTACT: 

3. 

FR/BTWN5 

2 . 

STH  GRD 

019 

T/ 

33 

(33  ) 

2222222232222222222222222222 

TOL5/ 

INTACT: 

3. 

FR/BTWN5 

3 . 

PREALG 

020 

T/ 

34 

(34  ) 

2222222223222222222222222222 

TOL5/ 

INTACT: 

3. 

FR/BTWN5 

4. 

ALGEBRA 

021 

T/ 

35 

(35  ) 

2222222222322222222222222222 

TOL5/ 

INTACT : 

3. 

FR/BTWN5 

5, 

INT/SEQ 

022 

T/ 

36 

(36  ) 

2222222222232222222222222222 

TOL5/ 

INTACT : 

3 . 

FR/BTWN5 

6. 

APPLIED 

023 

T/ 

37 

(37  ) 

2222222222223222222222222222 

TOL5/ 

INTACT : 

3 . 

FR/BTWN5 

7 , 

OTHER 

024 

T/ 

3S 

(3S  ) 

2222222222222322222222222222 

TOL5/ 

INTACT : 

3 . 

FR/BTWN5 

S. 

MISSING 

025 

T/ 

41 

(41  ) 

2222222222222211111112222222 

TOL5/ 

INTACT : 

4 . 

SML  TWN5 

1. 

NO  MATH 

026 

T/ 

42 

(42  ) 

2222222222222232222222222222 

TOL5/ 

INTACT : 

4 . 

SML  TWN5 

2 . 

STH  GRD 

027 

T/ 

43 

(43  ) 

2222222222222223222222222222 

TOL5/ 

INTACT : 

4 . 

SML  TWN5 

3. 

PREALG 

02S 

T/ 

44 

(44  ) 

2222222222222222322222222222 

TOL5/ 

INTACT : 

4 . 

SML  TWN5 

4. 

ALGEBRA 

029 

T/ 

45 

(45  ) 

2222222222222222232222222222 

TOL5/ 

INTACT : 

4 . 

SML  TWN5 

5. 

INT/SEQ 

030 

T/ 

46 

(46  ) 

2222222222222222223222222222 

TOL5/ 

INTACT : 

4. 

SML  TWN5 

6. 

APPLIED 

031 

T/ 

47 

(47  ) 

2222222222222222222322222222 

TOL5/ 

INTACT : 

4. 

SML  TWN5 

7. 

OTHER 

032 

T/ 

4S 

(4S  ) 

2222222222222222222232222222 

TOL5/ 

INTACT : 

4. 

SML  TWN5 

S. 

MISSING 

033 

T/ 

51 

(51  ) 

2222222222222222222221111111 

TOL5/ 

INTACT : 

5. 

RURAL 5 

1, 

NO  MATH 

034 

T/ 

52 

(52  ) 

2222222222222222222223222222 

TOL5/ 

INTACT : 

5 . 

RURAL 5 

2. 

STH  GRD 

035 

T/ 

53 

(53  ) 

2222222222222222222222322222 

TOL5/ 

INTACT : 

5. 

RURAL 5 

3. 

PREALG 

036 

T/ 

54 

(54  ) 

2222222222222222222222232222 

TOL5/ 

INTACT : 

5. 

RURAL 5 

4. 

ALGEBRA 

037 

T/ 

55 

(55  ) 

2222222222222222222222223222 

TOL5/ 

INTACT : 

5. 

RURAL5 

5. 

INT/SEQ 

03S 

T/ 

56 

(56  ) 

2222222222222222222222222322 

TOL5/ 

INTACT : 

5. 

RURAL 5 

6. 

APPLIED 

039 

T/ 

57 

(57  ) 

2222222222222222222222222232 

TOL5/ 

INTACT : 

; 5. 

RURAL 5 

7. 

OTHER 

040 

T/ 

5S 

(5S  ) 

2222222222222222222222222223 

TOL5/ 

INTACT : 

; 5. 

RURAL 5 

S. 

MISSING 

CONDITIONING 

VARIABLE  ID: 

BACK0066 

DESCRIPTION: 

INTERACTION:  SCHOOL  TYPE  BY 

MATH  COURSES 

TAKING 

THIS  YEAR 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


NOS,  SOS 
SCHT/ 

N/A 

INTERACTION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


24 

14 


001 

S/ 

11 

(11 

) 0101010101010101010101010101 

SCHT/ 

INTACT : 

1. 

PUBLIC 

1. 

NO  MATH 

002 

S/ 

12 

(12 

) -1000000000000-1000000000000 

SCHT/ 

INTACT : 

1. 

PUBLIC 

2 . 

STH  GRD 

003 

S/ 

13 

(13 

) 00-1000000000000-10000000000 

SCHT/ 

INTACT : 

1. 

PUBLIC 

3. 

PREALG 

004 

s/ 

14 

(14 

) 0000-1000000000000-100000000 

SCHT/ 

INTACT : 

1. 

PUBLIC 

4. 

ALGEBRA 

005 

S/ 

15 

(15 

) 000000-1000000000000-1000000 

SCHT/ 

INTACT : 

1. 

PUBLIC 

5. 

INT/SEQ 

006 

S/ 

16 

(16 

) 00000000-1000000000000-10000 

SCHT/ 

INTACT : 

1. 

•PUBLIC 

6. 

APPLIED 

007 

S/ 

17 

(17 

) 0000000000-1000000000000-100 

SCHT/ 

INTACT : 

1. 

PUBLIC 

7. 

OTHER 

OOS 

S/ 

IS 

(IS 

) 000000000000-1000000000000-1 

SCHT/ 

INTACT : 

1. 

PUBLIC 

S. 

MISSING 

009 

S/ 

21 

(21 

) -1-1-1-1-1-1-100000000000000 

SCHT/ 

INTACT : 

2. 

PRIVATE 

1. 

NO  MATH 

010 

S/ 

22 

(22 

) 0100000000000000000000000000 

SCHT/ 

INTACT : 

2. 

PRIVATE 

2. 

STH  GRD 

ERIC 
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oil 

S/ 

23 

(23 

012 

S/ 

24 

(24 

013 

S/ 

25 

(25 

014 

S/ 

26 

(26 

015 

S/ 

27 

(27 

016 

s/ 

28 

(28 

017 

S/ 

31 

(31 

018 

S/ 

32 

(32 

019 

s/ 

33 

(33 

020 

S/ 

34 

(34 

021 

S/ 

35 

(35 

022 

S/ 

36 

(36 

023 

S/ 

37 

(37 

024 

S/ 

38 

(38 

) 0001000000000000000000000000 
) 0000010000000000000000000000 
) 0000000100000000000000000000 
) 0000000001000000000000000000 
) 0000000000010000000000000000 
) 0000000000000100000000000000 
) 00000000000000-1-1-1-1-1-1-1 
) 0000000000000001000000000000 
) 0000000000000000010000000000 
) 0000000000000000000100000000 
) OOOOOOOOOOOOOOOOOOOOOIOOQOOO 
) 0000000000000000000000010000 
) 0000000000000000000000000100 
) 0000000000000000000000000001 


SCHT/ 

INTACT : 

2. 

PRIVATE 

3 . 

PREALG 

SCHT/ 

INTACT : 

2. 

PRIVATE 

4. 

ALGEBRA 

SCHT/ 

INTACT : 

2 . 

PRIVATE 

5. 

INT/SEQ 

SCHT/ 

INTACT : 

2 . 

PRIVATE 

6. 

APPLIED 

SCHT/ 

INTACT : 

2. 

PRIVATE 

7. 

OTHER 

SCHT/ 

INTACT : 

2. 

PRIVATE 

8. 

MISSING 

SCHT/ 

INTACT : 

3 . 

CATHOLIC 

1. 

NO  MATH 

SCHT/ 

INTACT : 

3. 

CATHOLIC 

2 . 

8TH  GRD 

SCHT/ 

INTACT : 

3. 

CATHOLIC 

3. 

PREALG 

SCHT/ 

INTACT : 

3 . 

CATHOLIC 

4. 

ALGEBRA 

SCHT/ 

INTACT : 

3 . 

CATHOLIC 

5. 

INT/SEQ 

SCHT/ 

INTACT : 

3. 

CATHOLIC 

6 . 

APPLIED 

SCHT/ 

INTACT : 

3. 

CATHOLIC 

7. 

OTHER 

SCHT/ 

INTACT : 

3 . 

CATHOLIC 

8. 

MISSING 

CONDITIONING  VARIABLE  ID:  BACK0059 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


INTERACTION: 

N12 

GEND/SEM 

N/A 

INTERACTION 


GENDER  BY  NUMBER  OF  SEMESTERS  MATH 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


10 

4 


001 

G/S 

11 

(11 

) 01010101 

GEND/SEM 

INTACT : 

1, 

. MALE 

1. 

SEMMATOl 

002 

G/S 

12 

(12 

) -1000000 

GEND/SEM 

INTACT : 

1, 

. MALE 

2 . 

SEMMAT02 

003 

G/S 

13 

(13  ; 

1 00-10000 

GEND/SEM 

INTACT : 

1, 

. MALE 

3 . 

SEMMAT03 

004 

G/S 

14 

(14  ; 

1 0000-100 

GEND/SEM 

INTACT : 

1, 

. MALE 

4 . 

SEMMAT04 

005 

G/S 

15 

(15  ] 

» 0000-100 

GEND/SEM 

INTACT : 

1, 

. MALE 

5. 

SEMMAT-? 

006 

G/S 

21 

(21  ; 

» -l-l-l-l 

GEND/SEM 

INTACT : 

2 , 

, FEMALE 

1. 

SEMMATOl 

007 

G/S 

22 

(22  : 

» 01000000 

GEND/SEM 

INTACT : 

2 . 

, FEMALE 

2 . 

SEMMAT02 

008 

G/S 

23 

(23  ] 

1 00010000 

. GEND/SEM 

INTACT : 

2 . 

, FEMALE 

3 . 

SEMMAT03 

009 

G/S 

24 

(24  ) 

1 00000100 

GEND/SEM 

INTACT : 

2 . 

, FEMALE 

4 . 

SEMMAT04 

010 

G/S 

25 

(25  ) 

1 00000001 

GEND/SEM 

INTACT : 

2 . 

, FEMALE 

5 . 

SEMMAT-? 

CONDITIONING 

VARIABLE  ID: 

BACK0061 

DESCRI PTION: 

INTERACTION: 

RACE/ETHNICITY  BY  NUMBER  OF 

SEMESTERS 

MATH 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TYPE  OF 

CONTRAST 

001 

R/S 

11 

(11 

002 

R/S 

12 

(12 

003 

R/S 

13 

(13 

004 

R/S 

14 

(14 

005 

R/S 

15 

(15 

006 

R/S 

21 

(21 

007 

R/S 

22 

(22 

008 

R/S 

23 

(23 

009 

R/S 

24 

(24 

010 

R/S 

25 

(25 

Oil 

R/S 

31 

(31 

012 

R/S 

32 

(32 

013 

R/S 

33 

(33 

014 

R/S 

34 

(34 

015 

R/S 

35 

(35 

016 

R/S 

41 

(41 

017 

R/S 

42 

(42 

018 

R/S 

43 

(43 

019 

R/S 

44 

(44 

020 

R/S 

45 

(45 

N12 

RACE /S EM 
N/A 

INTERACTION 

) 010101010101010101010101 
) -1000000-1000000-1000000 
) 00-1000000-1000000-10000 
) 0000-1000000-1000000-100 
) 000000-1000000-1000000-1 
) -1-1-1-10000000000000000 
) 010000000000000000000000 
) 000100000000000000000000 
) 000001000000000000000000 
) 000000010000000000000000 
) 00000000-1-1-1-100000000 
) 000000000100000000000000 
) 000000000001000000000000 
) 000000000000010000000000 
) 000000000000000100000000 
) 0000000000000000-1-1-1-1 
) 000000000000000001000000 
) 000000000000000000010000 
) 000000000000000000000100 
) 000000000000000000000001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


RACE /S EM 
RACE/ S EM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 
RACE/SEM 


20 

12 


INTACT ; 

1. 

WHl/AI/0 

1. 

SEMMATOl 

INTACT ; 

1. 

WHI/AI/0 

2 . 

SEMMAT02 

INTACT : 

1. 

WHI/AI/0 

3 . 

SEMMAT03 

INTACT : 

1. 

WHI/AI/0 

4. 

SEMMAT04 

INTACT ; 

1. 

WHI/AI/0 

5. 

SEMMAT-? 

INTACT ; 

2 . 

BLACK 

1. 

SEMMATOl 

INTACT ; 

2 . 

BLACK 

2 . 

SEMMAT02 

INTACT : 

2 . 

BLACK 

3 . 

SEMMAT03 

INTACT ; 

2 . 

BLACK 

4. 

SEMMAT04 

INTACT ; 

2 . 

BLACK 

5. 

SEMMAT-? 

INTACT ; 

3 . 

HISPANIC 

1. 

SEMMATOl 

INTACT ; 

3 . 

HISPANIC 

2 . 

SEMMAT02 

INTACT ; 

3 . 

HISPANIC 

3 . 

SEMMAT03 

INTACT ; 

3 . 

HISPANIC 

4. 

SEMMAT04 

INTACT : 

3 . 

HISPANIC 

5. 

SEMMAT-? 

INTACT : 

4 . 

ASIAN 

1. 

SEMMATOl 

INTACT : 

4 . 

ASIAN 

2 . 

SEMMAT02 

INTACT : 

4 . 

ASIAN 

3 . 

SEMMAT03 

INTACT : 

4. 

ASIAN 

4. 

SEMMAT04 

INTACT : 

4. 

ASIAN 

5. 

SEMMAT-? 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS  t 
CONDITIONING  VAR  LABEL: 


NAEP  ID: 

TYPE  OF 

CONTRAST 

001 

P/S 

11 

(11 

002 

P/S 

12 

(12 

003 

P/S 

13 

(13 

004 

P/S 

14 

(14 

005 

P/S 

15 

(15 

006 

P/S 

21 

(21 

007 

P/S 

22 

(22 

008 

P/S 

23 

(23 

009 

P/S 

24 

(24 

010 

P/S 

25 

(25 

Oil 

P/S 

31 

(31 

012 

P/S 

32 

(32 

013 

P/S 

33 

(33 

014 

P/S 

34 

(34 

015 

P/S 

35 

(35 

016 

P/S 

41 

(41 

017 

P/S 

42 

(42 

018 

P/S 

43 

(43 

019 

P/S 

44 

(44 

BACK0063 

INTERACTION: 

N12 

PARE/SEM 

N/A 

INTERACTION 


PARENTS'  EDUCATION  BY  NUMBER  OF  SEMESTERS  MATH 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


25 

16 


01010101010101010101010101010101  PARE/SEM 
-1000000-1000000-1000000-1000000  PARE/SEM 
00-1000000-1000000-1000000-10000  PARE/SEM 
0000-1000000-1000000-1000000-100  PARE/SEM 
000000-1000000-1000000-1000000-1  PARE/SEM 
-1-1-1-1000000000000000000000000  PARE/SEM 
OlOOOOOpOOOOOOOOOOOOOOOOOOOOOOOO  PARE/SEM 
00010000000000000000000000000000  PARE/SEM 
00000100000000000000000000000000  PARE/SEM 
00000001000000000000000000000000  PARE/SEM 
00000000-1-1-1-10000000000000000  PARE/SEM 
00000000010000000000000000000000  PARE/SEM 
00000000000100000000000000000000  PARE/SEM 
00000000000001000000000000000000  PARE/SEM 
00000000000000010000000000000000  PARE/SEM 
0000000000000000-1-1-1-100000000  PARE/SEM 
00000000000000000100000000000000  PARE/SEM 
00000000000000000001000000000000  PARE/SEM 
00000000000000000000010000000000  PARE/SEM 


INTACT : 

1. 

< HS 

1. 

SEMMATOl 

INTACT : 

1. 

< HS 

2 . 

SEMMAT02 

INTACT : 

1. 

< HS 

3 . 

SEMMAT03 

INTACT : 

1. 

< HS 

4. 

SEMMAT04 

INTACT : 

1. 

< HS 

5. 

SEMMAT-? 

INTACT : 

2 . 

HS  GRAD 

1. 

SEMMATOl 

INTACT : 

2. 

HS  GRAD 

2 . 

SEMMAT02 

INTACT : 

2. 

HS  GRAD 

3- 

SEMMAT03 

INTACT : 

2 . 

HS  GRAD 

4. 

SEMMAT04 

INTACT : 

2 . 

HS  GRAD 

5. 

SEMMAT-? 

INTACT : 

3- 

POST  HS 

1- 

SEMMATOl 

INTACT : 

3- 

POST  HS 

2 . 

SEMMAT02 

INTACT: 

3. 

POST  HS 

3 . 

SEMMAT03 

INTACT: 

3- 

POST  HS 

4. 

SEMMAT04 

INTACT: 

3. 

POST  HS 

5- 

SEMMAT-? 

INTACT : 

4. 

COL  GRAD 

1. 

SEMMATOl 

INTACT : 

4. 

COL  GRAD 

2 . 

SEMMAT02 

INTACT : 

4. 

COL  GRAD 

3 . 

SEMMAT03 

INTACT: 

4. 

COL  GRPiD 

4. 

SEMMAT04 

o 

ERIC 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


020 

P/S 

45 

(45 

021 

P/S 

51 

(51 

022 

P/S 

52 

(52 

023 

P/S 

53 

(53 

024 

P/S 

54 

(54 

025 

P/S 

55 

(55 

00000000000000000000000100000000  PARE/SEM  INTACT; 
000000000000000000000000-1-1-1-1  PARE/SEM  INTACT; 
00000000000000000000000001000000  PARE/SEM  INTACT: 
00000000000000000000000000010000  PARE/SEM  INTACT: 
00000000000000000000000000000100  PARE/SEM  INTACT: 
00000000000000000000000000000001  PARE/SEM  INTACT: 


4. 

COL  GRAD 

5. 

SEMMAT-? 

5. 

PARED-? 

1. 

SEMMATOl 

5. 

PARED-? 

2. 

SEMMAT02 

5. 

PARED-?  ■ 

3. 

SEMMAT03 

5. 

PARED-? 

4 . 

SEMMAT04 

5. 

PARED-? 

5. 

SEMMAT-? 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 


BACK0065 

INTERACTION: 

N12 


conditioning 

VAR  LABEL: 

TOL5/SEM 

NAEP  ID: 

N/A 

TYPE  OF 

CONTRAST : 

INTERACTION 

001 

T/S 

11 

(11 

) 01010101010101010101010101010101 

002 

T/S 

12 

(12 

) -1000000-1000000-1000000-1000000 

003 

T/S 

13 

(13 

) 00-1000000-1000000-1000000-10000 

004 

T/S 

14 

(14 

) 0000-1000000-1000000-1000000-100 

005 

T/S 

15 

(15 

) 000000- loooboo- 1000000-1000000-1 

006 

T/S 

21 

(21 

) -1-1-1-1000000000000000000000000 

007 

T/S 

22 

(22 

) 01000000000000000000000000000000 

OOS 

T/S 

23 

(23 

) 00010000000000000000000000000000 

009 

T/S 

24 

(24 

) 00000100000000000000000000000000 

010 

T/S 

25 

(25 

) 00000001000000000000000000000000 

Oil 

T/S 

31 

(31 

) 00000000-1-1-1-10000000000000000 

012 

T/S 

32 

(32 

) 00000000010000000000000000000000 

013 

T/S 

33 

(33 

) 00000000000100000000000000000000 

014 

T/S 

34 

(34 

) 00000000000001000000000000000000 

015 

T/S 

35 

(35 

) 00000000000000010000000000000000 

016 

T/S 

41 

(41 

) 0000000000000000-1-1-1-100000000 

017 

T/S 

42 

(42 

) 00000000000000000100000000000000 

OIS 

T/S 

43 

(43 

) 00000000000000000001000000000000 

019 

T/S 

44 

(44 

) 00000000000000000000010000000000 

020 

T/S 

45 

(45 

) 00000000000000000000000100000000 

021 

T/S 

51 

(51 

) 000000000000000000000000-1-1-1-1 

022 

T/S 

52 

(52 

) 00000000000000000000000001000000 

023 

T/S 

53 

(53 

) 00000000000000000000000000010000 

024 

T/S 

54 

(54 

) 00000000000000000000000000000100 

025 

T/S 

55 

(55 

) 00000000000000000000000000000001 

CONDITIONING 

VARIABLE  IE 

I:  BACK0067 

DESCRIPTION: 

INTERACTION:  SCHOOL  TYPE  BY  NUME 

GRADES /ASSESSMENTS ; 

N12 

CONDITIONING 

VAR  LABEL: 

SCHT/SEM 

NAEP  ID; 

N/A 

TYPE  OF 

CONTRAST: 

INTERACTION 

001 

S/S 

11 

(11 

) 0101010101010101 

002 

S/S 

12 

(12 

) -1000000-1000000 

003 

S/S 

13 

(13 

) 00-1000000-10000 

004 

S/S 

14 

(14 

) 0000-1000000-100 

005 

S/S 

15 

(15 

) 000000-1000000-1 

006 

S/S 

21 

(21 

) -1-1-1-100000000 

007 

S/S 

22 

(22 

) 0100000000000000 

OOS 

S/S 

23 

(23 

) 0001000000000000 

009 

S/S 

24 

(24 

) 0000010000000000 

010 

S/S 

25 

(25 

) 0000000100000000 

Oil 

S/S 

31 

(31 

> 00000000-1-1-1-1 

012 

s/s 

32 

(32 

) 0000000001000000 

013 

s/s 

33 

(33 

) 0000000000010000 

014 

s/s 

34 

(34 

) 0000000000000100 

015 

s/s 

35 

(35 

) 0000000000000001 

TYPE  OF  LOCALE  (5  CATEGORIES)  BY  NUMBER  OF  SEMESTERS  MATH 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  25 

NUMBER  OF  INDEPENDENT  CONTRASTS:  16 


TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 

TOL5/SEM 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS : 15 

NUMBER  OF  INDEPENDENT  CONTRASTS:  8 


SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 


INTACT : 

1. 

BIG 

CTY5 

1. 

SEMMATOl 

INTACT : 

1. 

BIG 

CTY5 

2 . 

SEMMAT02 

INTACT : 

1. 

BIG 

CTY5 

3 . 

SEMMAT03 

INTACT : 

1. 

BIG 

CTY5 

4. 

SEMMAT04 

INTACT : 

1. 

BIG 

CTY5 

5. 

SEMMAT-? 

INTACT: 

2 . 

MID 

CTY5 

1 . 

SEMMATOl 

INTACT : 

2 . 

MID 

CTY5 

2 . 

SEMMAT02 

INTACT : 

2 . 

MID 

CTY5 

3 . 

SEMMAT03 

INTACT : 

2 . 

MID 

CTY5 

4. 

SEMMAT04 

INTACT: 

2 . 

MID 

CTY5 

5 . 

SEMMAT-? 

INTACT: 

3 . 

FR/BTWN5 

1 . 

SEMMATOl 

INTACT: 

3 . 

FR/BTWN5 

2 . 

SEMMAT02 

INTACT : 

3 . 

FR/BTWN5 

3 . 

SEMMAT03 

INTACT : 

3 . 

FR/BTWN5 

4 . 

SEMMAT04 

INTACT: 

3 . 

FR/BTWN5 

5. 

SEMMAT-? 

INTACT: 

4 . 

SML 

TWN5 

1. 

SEMMATOl 

INTACT: 

4. 

SML 

TWN5 

2 . 

SEMMAT02 

INTACT: 

4. 

SML 

TWN5 

3 . 

SEMMAT03 

INTACT : 

4 . 

SML 

TWN5 

4 . 

SEMMAT04 

INTACT : 

4 . 

SML 

TWN5 

5. 

SEMMAT-? 

INTACT : 

5. 

RURAL5 

1. 

SEMMATOl 

INTACT: 

5. 

RURAL5 

2 . 

SEMMAT02 

INTACT : 

5. 

RURAL 5 

3 . 

SEMMAT03 

INTACT : 

5. 

RURAL5 

4. 

SEMMAT04 

INTACT : 

5 . 

RURAL5 

5. 

SEMMAT-? 

INTACT ; 

1. 

PUBLIC 

1. 

SEMMATOl 

INTACT ; 

1. 

PUBLIC 

2 . 

SEMMAT02 

INTACT; 

1. 

PUBLIC 

3 . 

SEMMAT03 

INTACT : 

1. 

PUBLIC 

4. 

SEMMAT04 

INTACT ; 

1. 

PUBLIC 

5. 

SEMMAT-? 

INTACT ; 

2 . 

PRIVATE 

1. 

SEMMATOl 

INTACT ; 

2 . 

PRIVATE 

2 . 

SEMMAT02 

INTACT; 

2 . 

PRIVATE 

3 . 

SEMMAT03 

INTACT : 

2 . 

PRIVATE 

4 . 

SEMMAT04 

INTACT : 

2 . 

PRIVATE 

5. 

SEMMAT-? 

INTACT ; 

3 . 

CATHOLIC 

1. 

SEMMATOl 

INTACT ; 

3 . 

CATHOLIC 

2 . 

SEMMAT02 

INTACT ; 

3 . 

CATHOLIC 

3 . 

SEMMAT03 

INTACT ; 

3 . 

CATHOLIC 

4. 

SEMMAT04 

INTACT ; 

3 . 

CATHOLIC 

5. 

SEMMAT-? 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

■ NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0071 
SAMPLE  TYPE 
N04,  S04,  NOS, 

SUB SAMP 
CLASS 


SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  SAMP  Si  (01 

002  SAMP  S2  (02 

003  SAMP  S3  (03 


) 00 
) 10 
) 01 


SAMPLE  SI 
SAMPLE  S2 
SAMPLE  S3 


CONDITIONING  VARIABLE  ID:  BACK0072 


DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


INTERACTION: 
N04,  S04,  NOS 
/RAC 


SAMPLE  BY  RACE/ETHNICITY 
, SOS,  N12 


NAEP 

TYPE 

ID: 

OF 

CONTRAST : 

N/A 

INTERACTION 

001 

/R 

11 

. (11 

) 010101010101 

002 

/R 

12 

(12 

) -10000-10000 

003 

/R 

13 

(13 

) 00-10000-100 

004 

/R 

14 

(14 

) 0000-10000-1 

005 

/R 

21 

(21 

> -1-1-1000000 

006 

/R 

22 

(22 

) 010000000000 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


12 

6 


/RAC 

INTACT 

1. 

SAMP 

SI 

1. 

WHI/AI/0 

/RAC 

INTACT 

1. 

SAMP 

SI 

2 . 

BLACK 

/RAC 

INTACT 

1. 

SAMP 

SI 

3 . 

HISPANIC 

/RAC 

INTACT 

1. 

SAMP 

SI 

4. 

ASIAN  ■ 

/RAC 

INTACT 

2. 

SAMP 

S2 

1. 

WHI/AI/0 

/RAC 

INTACT 

2. 

SAMP 

S2 

2. 

BLACK 

ERIC 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


007 

/R 

23 

(23 

OOS 

/R 

24 

(24 

009 

/R 

31 

(31 

010 

/R 

32 

(32 

Oil 

/R 

33 

(33 

012 

/R 

34 

(34 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 000100000000 
) 000001000000 
) 000000-1-1-1 
) 000000010000 
) 000000000100 
) 000000000001 

BACK0073 

REPORTING  SAMPLE 
N04,  S04,  NOS,  SOS,  N12 

RPTSAMP 

CLASS 


/RAC 

INTACT;  2. 

SAMP 

S2 

3. 

HISPANIC 

/RAC 

INTACT;  2. 

SAMP 

S2 

4. 

ASIAN 

/RAC 

INTACT;  3. 

SAMP 

S3 

1. 

WHI/AI/0 

/RAC 

INTACT:  3. 

SAMP 

S3 

2 . 

BLACK 

/RAC 

INTACT;  3. 

SAMP 

S3 

3. 

HISPANIC 

/RAC 

INTACT;  3. 

SAMP 

S3 

4. 

ASIAN 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

NUMBER  OF  INDEPENDENT  CONTRASTS;  1 


001  RPTSAMP  (01  ) 0 

002  RPT  NO  (02  } 1 


YES 

NO 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

BACK0074 
HOW  OFTEN 

USE  A HOME  COMPUTER  FOR  SCHOOLWORK? 

GRADES /ASSESSMENTS : 

N04,  S04, 

NOS,  SOS,  N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

B009301 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS ; 

001  B009301A  (01  ! 

1 00000 

ALMOST  EVERY  DAY 

002  B009301B  (02  ] 

1 10000 

ONCE  OR  TWICE  A WEEK 

003  B009301C  (03  ] 

1 01000 

ONCE  OR  TWICE  A MTH 

004  B009301D  (04  ] 

1 00100 

NEVER  OR  HARDLY  EVER 

005  B009301E  (05  ) 

t 00010 

NO  COMPUTER  AT  HOME 

006  B009301M  (M  ) 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

BACK0075 

HOW  SAFE  DO  YOU  FEEL  AT  SCHOOL? 

GRADES /ASSESSMENTS ; 

N04,  S04, 

NOS,  SOS.  N12 

CONDITIONING  VAR  LABEL; 
NAEP  ID: 

B009401 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  B009401A  (01  ) 0000 

002  B009401B  (02  ) 1000 

003  B009401C  (03  ) 0100 

004  B009401D  (04  ) 0010 

005  B009401M  (M  ) 0001 


VERY  SAFE 
SOMEWHAT  SAFE 
SOMEWHAT  UNSAFE 
VERY  UNSAFERDLY  EVER 
MISSING 


BACK0076 

DOES  YOUR  STEP /MOTHER  WORK  AT  A JOB  FOR  PAY? 

N04,  S04 

B009501  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

class  number  of  INDEPENDENT  CONTRASTS; 

001  B009501A  (01  ) 000 

002  B009501B  (02  ) 100 

003  B009501C  (03  ) 010 

004  B009501M  (M  ) 001 


YES 

NO 

DON'T  LIVE  W/  MOTHER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION:  ' 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


BACK0077 

DOES  YOUR  STEP /FATHER  WORK  AT  A JOB  FOR  PAY? 

N04,  S04 

B009502  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  B009502A  (01  ) 000 

002  B009502B  (02  ) 100 

003  B009502C  (03  ) 010 

004  B009502M  (M  ) 001 


YES 

NO 

DON'T  LIVE  W/  FATHER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST: 


SUBJ0002 

HOW  OFTEN  DO  YOU  DO  MATH  PROBLEMS  FROM  TEXTBOOK? 

N04,  S04,  NOS,  SOS,  N12 

MS12701  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  MS12701A  (01  ) 0000 

002  MS12701B  (02  ) 1000 

003  MS12701C  (03  ) 0100 

004  MS12701D  (04  ) 0010 

005  MS12701M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0003 

HOW  OFTEN  DO  YOU  DO  MATH  PROBLEMS  ON  WORK  SHEETS? 

N04,  S04,  NOS,  SOS,  N12 

MS12702  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


6 

5 


5 

4 


4 

3 


4 

3 


5 

4 


5 

4 


O ^88  . 

ERIC 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


001  M812702A  (01  ) 0000 

002  M812702B  (02  ) 1000 

003  M812702C  (03  ) 0100 

004  M812701D  (04  ) 0010 

005  M812701M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID : 

TYPE  OF  CONTRAST: 


SUBJ0004 

HOW  OFTEN  SOLVE  MATH  PROBLEMS  WITH  PARTNER /GROUP? 

N04,  S04,  N08,  S08,  N12 

M812703  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

M812703A 

(01 

) 0000 

002 

M812703B 

(02 

) 1000 

003 

M812703C 

(03 

) 0100 

004 

M812703D 

(04 

) 0010 

005 

M812703M 

(M 

) 0001 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


5 

4 


CONDITIONING  VARIABLE  ID:  SUBJ0005 

DESCRIPTION:  HOW  OFTEN  IN  MATH  WORK  W/  RULERS,  GEOM  SHAPES? 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

N04,  S04 

NAEP  ID; 

M812704 

TYPE  OF  CONTRAST; 

CLASS 

001  M812704A  (01 

) 0000 

002  M812704B  (02 

) 1000 

003  M812704C  (03 

) 0100 

004  M812704D  (04 

) 0010 

005  M812704M  (M 

) 0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  5 

NUMBER  OF  INDEPENDENT  CONTRASTS:  4 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  M812705A  (01  ) 0000 

002  M812705B  (02  > 1000 

003  M812705C  (03  > 0100 

004  M812705D  (04  > 0010 

005  M812705M  (M  > 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK  ‘ 

ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING  RDLY  EVER 


SUBJ0006 

HOW  OFTEN  WRITE  ABOUT  SOLVING  A MATH  PROBLEM? 
N04,  S04,  N08,  S08,  N12 

M812705 
CLASS 


CONDITIONING  VARIABLE 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  M812706A  (01  ) 0000 

002  M812706B  (02  ) 1000 

003  M812706C  (03  ) 0100 

004  M812706D  (04  ) 0010 

005  M812706M  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


ID:  SUBJ0007 

HOW  OFTEN  DO  YOU  TAKE  MATHEMATICS  TESTS? 
N04,  S04,  N08,  S08,  N12 

M812706 
CLASS 


CONDITIONING  VARIABLE 
DESCRIPTION; 

GRADES /ASSESSMENTS; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


001  M812707A  (01  ) 0000 

002  M812707B  (02  ) 1000 

003  M812707C  (03  ) 0100 

004  M812707D  (04  ) 0010 

005  M812707M  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


ID;  SUBJ0008 

HOW  OFTEN  DO  YOU  TALK  TO  THE  CLASS  RE  MATH  WORK? 
N04,  S04,  N08.  S08,  N12 

M812707 
CLASS 


CONDITIONING  VARIABLE 
DESCRIPTION; 

GRADES /ASSES  SMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


001 

M812708A 

(01 

) 0000 

002 

M812708B 

(02 

) 1000 

003 

M812708C 

(03 

) 0100 

004 

M812708D 

(04 

) 0010 

005 

M812708M 

(M 

) 0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


ID:  SUBJ0009 

HOW  OFTEN  DO  10  OR  MORE  MATH  PROBLEMS  BY  YOURSELF? 
N04,  S04.  N08,  S08.  N12 

M812708 
CLASS 


CONDITIONING  VARIABLE  ID:  SUBJOOlO 

DESCRIPTION;  HOW  OFTEN  DISCUSS  MATH  PROBLEMS  W/  OTHER  STUDENTS? 


5 

4 


5 

4 


5 

4 


• 


BEST  COPY  AVAILABLE 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


N04,  S04,  NOS,  SOS,  N12 

MS12709 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

MS12709A 

(01 

) 0000 

002 

MS12709B 

(02 

) 1000 

003 

MS12709C 

(03 

) 0100 

004 

MS12709D 

(04 

) 0010 

005 

MS12709M 

(M 

) 0001 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  MS12710 

TYPE  OF  CONTRAST:  CLASS 

001  MS12710A  (01  ) 0000 

002  MS12710B  (02  ) 1000 

003  MS12710C  (03  ) 0100 

004  MS12710D  (04  ) 0010 

005  MS12710M  (M  ) 0001  . 


SUBJOOll 

HOW  OFTEN  DO  YOU  USE  A COMPUTER  FOR  MATH? 
N04,  S04,  NOS,  SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  MS12711 

TYPE  OF  CONTRAST:  CLASS 

001  MS12711A  (01  ) 0000 

002  MS12711B  (02  ) 1000 

003  MS12711C  (03  ) 0100 

004  MS12711D  (04  ) 0010 

005  MS12711M  (M  ) 0001 


SUBJ0012 

HOW  OFTEN  DO  YOU  USE  A CALCULATOR  FOR  MATH? 
N04,  S04,  NOS,  SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0013 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

TYPE  OF  CONTRAST: 


DO  YOU  HAVE  CALCULATOR  TO  USE  WITH  MATH  HOMEWORK? 
N04,  S04,  NOS,  SOS,  N12 


MS11201 

CLASS 


001  MS11201Y  (01 

002  MS11201N  (02 

003  MS11201M  (M 


) 


00 

10 

01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NODECIDED 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  MS12001A  (01 

002  MS12001B  (02 

003  MS12001C  (03 

004  MS12001D  (04 

005  MS12001M  (M 


SUBJ0014 

HOW  OFTEN  USE  A CALCULATOR  FOR  MATH  CLASSWORK? 
N04,  S04,  NOS,  SOS,  N12 


MS12001 

CLASS 

0000 

1000 

0100 

0010 

0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  MS12002A  (01 

002  MS12002B  (02 

003  MS12001C  (03 

004  MS12001D  (04 

005  MS12001M  (M 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


SUBJ0015 

HOW  OFTEN  USE  A CALCULATOR  FOR  MATH  HOMEWORK? 
N04,  S04,  NOS,  SOS,  N12 


MS12002 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04 

NAEP  ID: 

MS12003 

TYPE  OF  CONTRAST; 

CLASS 

001  MS12003A  (01 

) 0000 

002  MS12003B  (02 

) 1000 

003  MS12003C  (03 

) 0100 

004  MS12003D  (04 

) 0010 

005  MS12003M  (M 

) 0001 

SUBJ0016 

HOW  OFTEN  USE  A CALCULATOR  FOR  MATH  TESTS/QUIZZES? 
S,  SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


5 

4 


5 

4 


5 

4 


3 

2 


5 

4 


5 

4 


O 90 

ERIC 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


M812101 


AST: 

CLASS 

(01  ) 

0000000 

(02  ) 

1000000 

(03  ) 

0100000 

(04  ) 

0010000 

(05  ) 

0001000 

(06  ) 

0000100 

(07  ) 

0000010 

(M  ) 

0000001 

VARIABLE  ID: 

SUBJ0018 

SUBJ0017 

HOW  MUCH  TIME  SPEND  DAILY  ON  MATH  HOMEWORK? 

N04,  S04,  NOS,  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  TAKING  THIS  YEAR 
NONE 

15  MINUTES 
30  MINUTES 
45  MINUTES 
ONE  HOUR 

MORE  THAN  ONE  HOUR 
MISSING 


DESCRIPTION:  GET  MATH 
GRADES /ASSESSMENTS:  N04 , S04 
CONDITIONING  VAR  LABEL: 

NAEP  ID:  MS11401 
TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  MS11401Y  <01  ) 00 

002  MS11401N  (02  ) 10 

003  MS11401M  (M  ) 01 


YES 

NODECIDED 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0019 

AGREE/DISAGREE:  l'  LIKE  MATH 
N04,  S04 

MSlllOl 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  MSlllOlA  (01  ) 000 

002  MSlllOlB  (02  ) 100 

003  MSlllOlC  (03  ) 010 

004  MSlllOlM  (M  ) 001 


AGREE 

undecided 

DISAGREE  OR  LESS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0020 

AGREE/DISAGREE:  I AM  GOOD  AT  MATH 
N04,  S04 


MS11103 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  MS11103A  (01  ) 000 

002  MS11103B  (02  ) 100 

003  MS11103C  (03  ) 010 

004  MS11103M  (M  ) 001 


AGREE 
UNDECIDED 
DISAGREE  OR  LESS 

missing 


8 

7 


3 

2 


4 

3 


4 

3 


ERIC 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0021 

HOW  MUCH  AGREE-UNDERSTAND  MOST  OF  MATH  CLASS? 
N04,  S04 


M811106 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811106A  (01  ) 000 

002  M811106B  (02  ) 100 

003  M811106C  (03  ) 010 

004  M811106M  (M  ) 001 


AGREE 
UNDECIDED 
DISAGREE  OR  LESS 
MISSING 


4 

3 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0022 

HOW  MUCH  AGREE -ONLY  1 CORRECT 
N04,  S04 


WAY  TO  SOLVE  PROB? 


M811109 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811109A  (01  ) 000 

002  M811109B  (02  ) 100 

003  M811109C  (03  ) 010 

004  M811109M  (M  ) 001 


AGREE 
UNDECIDED 
DISAGREE  OR  LESS 
MISSING 


4 

3 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 


HOW  MUCH  AGREE -LEARNING  MATH  IS  MEMORIZING  FACTS? 
N04,  S04 


M811107 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001 

M811107A 

(01 

) 000 

002 

M811107B 

(02 

) 100 

003 

M811107C 

(03 

) 010 

004 

M811107M 

(M 

) 001 

AGREE 
UNDECIDED 
DISAGREE  OR  LESS 
MISSING 


4 

3 


BEST  COPY  AVAILABLE  491 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0024 

AGREE/DISAGREE:  MATH  USED  FOR  SOLVING  PROBLEMS 
N04,  S04 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
^^SS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811105A  (01  ) 000 

002  M811105B  (02  ) 100 

003  M811105C  (03  ) 010 

004  M811105M  (M  ) 001 


AGREE 
UNDECIDED 
DISAGREE  OR  LESS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0025 

HOW  MUCH  AGREE-IF  CHOICE  I WOULDN'T  STUDY  MORE? 
N04,  S04 


M811108 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811108A  (01  ) 000 

002  M811108B  (02  ) 100 

003  M811108C  (03  ) 010 

004  M811108M  (M  ) 001 


AGREE 
UNDECIDED 
DISAGREE  OR  LESS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0026 

HOW  MUCH  AGREE- EVERYONE  CAN  DO  WELL  IF  TRY? 
N04,  S04 


M811110 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811110A  (01  ) 000 

002  M811110B  (02  ) 100 

003  M811110C  (03  ) 010 

004  M811110M  (M  ) 001 


AGREE 
UNDECIDED 
DISAGREE  OR  LESS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  MMOOIOIA  (01  ) 0000 

002  MMOOIOIB  (02  ) 1000 

003  MMOOIOIC  (03  ) 0100 

004  MMOOIOID  (04  ) 0010 

005  MMOOIOIM  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  ALL 
MORE  THAN  HALF 
ABOUT  HALF 
LESS  THAN  HALF 
MISSING 


SUBJ0027 

ABOUT  HOW  MANY  QUESTIONS  RIGHT  ON  TEST? 
N04,  S04,  N08,  S08,  N12 

MMOOlOl 
CLASS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SUBJ0028 

HOW  HARD  TEST  COMPARED  TO  THOSE  IN  SCHOOL? 
N04,  S04,  N08,  S08,  N12 


MM00201 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  MM00201A  (01  ) 0000 

002  MM00201B  (02  ) 1000 

003  MM00201C  (03  ) 0100 

004  MM00201D  (04  ) 0010 

005  MM00201M  (M  ) 0001 


MUCH  HARDER 
HARDER  THAN  OTHERS 
ABOUT  THE  SAME 
EASIER  THEN  OTHES 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SUBJ0029 

HOW  HARD  DID  YOU  TRY  ON  TEST  COMPARED  TO  OTHERS? 

N04,  S04,  N08,  S08,  N12 

^00301  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPa«)ENT  CONTRASTS: 


001  MM00301A  (01  ) 0000 

002  MM00301B  (02  ) 1000 

003  MM00301C  (03  ) 0100 

004  MM00301D  (04  ) 0010 

005  MM00301M  (M  ) 0001 


MUCH  HARDER 
HARDER  THAN  OTHERS 
ABOUT  AS  HARD 
NOT  AS  HARD  AS  OTHER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0030 

HOW  IMPORTANT  WAS  IT  YOU  DO  WELL  ON  THIS  TEST? 

N04,  S04,  N08,  S08,  N12 

^00401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

MM00401A 

(01 

) 0000 

002 

MM00401B 

(02 

) 1000 

003 

MM00401C 

(03 

) 0100 

004 

MM00401D 

(04 

) 0010 

005 

MM00401M 

(M 

) 0001 

VERY  IMPORTANT 
IMPORTANT 

SOMEWHAT  IMPORTANT 
NOT  VERY  IMPORTANTER 
MISSING 


4 

3 


4 

3 


4 

3 


5 

4 


5 

4 


5 

4 


5 

4 


o n 

ERIC 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


ERIC 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0031 

HOW  OFTEN  HAD  TO  WRITE  LONG  ANSWERS  TO  QSTS? 

N04,  S04,  NOS,  SOS,  N12  . 

MM00501  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  independent  CONTRASTS: 


001  MM00501A  (01 

) 

0000 

AT  LEAST  ONCE  A WEEK 

002  MM00501B  (02 

) 

1000 

ONCE  OR  TWICE  A MNTH 

003  MM00501C  (03 

) 

0100 

ONCE  OR  TWICE  A YEAR 

004  MM00501D  (04 

) 

0010 

NEVERERY  IMPORTANTER 

005  MM00501M  (M 

) 

0001 

MISSING 

CONDITIONING  VARIABLE  ID: 

BACK0078 

DESCRIPTION: 

DESCRIBE  YOUR  OVERALL 

GRADES  SINCE  6TH  GRADE 

GRADES /ASSESSMENTS:  . 

N08,  S08 

CONDITIONING  VAR  LABEL: 

MaPP  TD* 

B009701 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  B009701A  (01 

) 

000000 

MOSTLY  A'S 

002  B009701B  (02 

) 

100000 

MOSTLY  B'S 

003  B009701C  (03 

) 

010000 

MOSTLY  C'S 

004  B009701D  (04 

) 

001000 

MOSTLY  D'S 

005  B009701E  (05 

) 

000100 

MOSTLY  BELOW  D'S 

006  B009701F  (06 

) 

000010 

CLASSES  NOT  GRADED 

007  B009701M  (M 

) 

000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

BACK0079 

DESCRIPTION: 

HOW  FAR  IN 

SCHOOL  DO  YOU 

THINK  YOU  WILL  GO? 

GRADES/ASSESSMENTS : 

N08,  S08 

CONDITIONING  VAR  LABEL: 

MaPP  TD • 

B009801 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  NO  HS  (01 

) 

000000 

NOT  FINISH  HS 

002  GRAD  HS  (02 

) 

100000 

GRADUATE  HS 

003  ED  > HS  (03 

) 

010000 

SOME  ED  PAST  HS 

004  GRAD  CLG  (04 

) 

001000 

GRADUATE  COLLEGE 

005  GRAD  SCH  (05 

) 

000100 

GO  GRADUATE  SCHOOL 

006  IDK  (06 

) 

000010 

I DON'T  KNOWGRADED 

007  MISSING  (M 

) 

000001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

BACK0080 

DESCRIPTION: 

DOES  YOUR 

STEP/MOTHER 

WORK  AT  A JOB  FOR  PAY? 

GRADES /ASSESSMENTS : 

N08,  S08, 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

B009601 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  B009601A  (01 

) 

1 0000 

YES,  FULL-TIME 

002  B009601B  (02 

) 

1 1000 

YES,  PART-TIME 

003  B009601C  (03 

] 

1.  0100 

NO 

004  B009601D  (04 

] 

1 0010 

DON'T  LIVE  W/  MOTHER 

005  B009601M  (M 

I 0001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

BACK0081 

DESCRIPTION: 

DOES  YOUR 

STEP/ FATHER 

WORK  AT  A JOB  FOR  PAY? 

GRADES /ASSESSMENTS : 

N08,  S08, 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

B009602 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  B009601A  (01 

) 0000 

YES,  FULL-TIME 

002  B009601B  (02 

) 1000 

YES,  PART-TIME 

003  B009601C  (03 

) 0100 

NO 

004  B009601D  (04 

) 0010 

DON'T  LIVE  W/  FATHER 

005  B009601M  (M 

) 0001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SUBJ0032 

DESCRIPTION: 

WORK  W/  MEAS.  INSTRUMENTS/GEOM  SOLIDS  FOR  MATH? 

GRADES/ ASSESSMENTS : 

N08,  S08. 

N12 

rnMniTIONING  VAR  LABEL: ■ 

NAEP  ID: 

M812712 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  M812712A  (01 

) 0000 

ALMOST  EVERY  DAY 

002  M812712B  (02 

) 1000 

ONCE  OR  TWICE  A WEEK 

003  M812712C  (03 

) 0100 

ONCE  OR  TWICE  MONTH 

004  M812712D  (04 

) 0010 

NEVER  OR  HARDLY  EVER 

005  M812712M  (M 

) 0001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SUBJ0033 

DESCRIPTION: 

HOW  OFTEN 

WRITE  REPORTS 

OR  DO  MATH  PROJECTS? 

GRADES /ASSESSMENTS : 

N08,  S08, 

N12 

CONDITIONING  VAR  LABEL: 


■!EST  COPY  AVAILABLE 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


NAEP  ID:  M812713 

TYPE  OF  CONTRAST;  CLASS 

001  M812713A  (01  ) 0000 

002  M812713B  (02  ) lOOO 

003  M812713C  (03  ) 0100 

004  M812713D  (04  ) 0010 

005  M812713M  (M  ) 0001 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  ISABEL: 

NAEP  ID:  . M812201 

TYPE  OF  CONTRAST:  CLASS 

001  M812201Y  (01  ) 00 

002  M812201N  (02  ) 10 

003  M812201M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING  RDLY  EVER 


SUBJ0034 

IS  THERE  A PORTFOLIO  W/  YOUR  MATH  WORK  IN  IT*? 
N08,  S08,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0035 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL 
NAEP  ID: 

TYPE  OF  CONTRAST: 


DO  YOU  USE  A SCIENTIFIC  CALCULATOR  FOR  MATH  WORK‘=' 
N08,  S08,  N12 


M812301 

CLASS 


001  M812301Y  (01 

002  M812301N  (02 

003  M812301M  (M 


) 00 
) 10 
) 01 


CONDITIONING  VARIABLE  ID; 

SUBJ0036 

DESCRIPTION: 

DO  YOU  USE  A 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

N08,  S08,  N12 

NAEP  ID: 

M812401 

TYPE  OF  CONTRAST: 

CLASS 

001  M812401Y  (01 

) 00 

002  M812401N  (02 

) 10 

003  M812401M  (M 

) 01 

CONDITIONING  VARIABLE  ID: 

SUBJ0037 

DESCRIPTION: 

WHAT  KIND  OF 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08,  S08 

NAEP  ID: 

MATEXP 

TYPE  OF  CONTRAST: 

CLASS 

001  NO  MATH  (01 

) 000000000 

002  BASIC  (02  ] 

1 100000000 

003  APPLIED  (03  ] 

1 010000000 

004  PREALG  (04  ] 

1 001000000 

005  ALG  1 (05  1 

1 000100000 

006  GEOMETRY  (06  ] 

1 000010000 

007  INT/SEQ  (07  ) 

1 000001000 

008  OTHER  (08  ) 

1 000000100 

009  IDK  (09  ) 

1 000000010 

010  missing  (M  ) 

000000001 

CONDITIONING  VARIABLE  ID: 

SUBJ0038 

DESCRIPTION: 

DO  YOU  AGREE: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08,  S08,  N12 

NAEP  ID: 

M810701 

TYPE  OF  CONTRAST: 

CLASS 

001  M812701A  (01  ) 

00000 

002  M812701B  (02  ) 

10000 

003  M812701C  (03  ) 

01000 

004  M812701D  (04  ) 

00100 

005  M812701E  (05  ) 

00010 

006  M812701M  (M  ) 

00001 

CONDITIONING  VARIABLE  ID: 

SUBJ0039 

DESCRIPTION: 

DO  YOU  AGREE: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08.  S08,  N12 

NAEP  ID: 

M810703 

TYPE  OF  CONTRAST: 

CLASS 

001  M810701A  (01  ) 

00000 

002  M810701B  (02  ) 

10000 

003  M810701C  (03  ) 

01000 

004  M810701D  (04  ) 

00100 

005  M810701E  (05  ) 

00010 

006  M810701M  (M  ) 

00001 

I LIKE  MATH 


I AM  GOOD  IN  MATH 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

NOT  EXPECT  TO  TAKE 
BAS IC , GEN , BUS INESS 
APPLIED  MATH 
PREALGEBRA 
ALGEBRA  1/ELEM  ALG 
GEOMETRY 

INTERGATED/SEQUENTAL 
OTHER  MATH  CLASS 
I DON'T  KNOW  ' 

MISSING  W 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


10 

9- 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


ERIC 


CONDITIONING  VARIABLE  ID: 

SUBJ0040 

DESCRIPTION: 

AGREE /DISAGREE: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS,  N12 

NAEP  ID: 

M810707 

TYPE  OF  CONTRAST: 

CLASS 

001  MS10707A  (01  ) 

00000 

002  MS10707B  (02  ) 

10000 

003  MS10707C  (03  ) 

01000 

004  MS10707D  (04  ) 

00100 

005  MS10707E  (05  ) 

00010 

006  MS10707M  (M  ) 

00001 

CONDITIONING  VARIABLE  ID: 

SUBJ0041 

DESCRIPTION: 

AGREE /DISAGREE: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS,  N12 

NAEP  ID: 

M810709 

TYPE  OF  CONTRAST: 

CLASS 

001  MS10709A  (01  ) 

> 00000 

002  MS10709B  (02  ) 

1 10000 

003  M810709C  (03  ) 

1 01000 

004  M810709D  (04  ) 

I 00100 

005  M810709E  (05  ) 

1 00010 

006  M810709M  (M  ) 

1 00001 

CONDITIONING  VARIABLE  ID: 

SUBJ0042 

DESCRIPTION: 

AGREE/ DISAGREE: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS,  N12 

NAEP  ID: 

M810708 

TYPE  OF  CONTRAST; 

CLASS 

001  M810708A  (01  1 

1 00000 

002  M810708B  (02  ] 

1 10000 

003  M810708C  (03 

1 01000 

004  M810708D  (04 

) 00100 

005  M810708E  (05 

) 00010 

006  M810708M  (M 

) 00001 

CONDITIONING  VARIABLE  ID: 

SUBJ0043 

DESCRIPTION: 

AGREE /DISAGREE 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS,  N12 

NAEP  ID: 

M810710 

TYPE  OF  CONTRAST: 

CLASS 

001  M810710A  (01 

) 00000 

002  M810710B  (02 

) 10000 

003  M810710C  (03 

) 01000 

004  M810710D  (04 

) 00100 

005  M810710E  (05 

) 00010 

006  M810710M  (M 

) 00001 

CONDITIONING  VARIABLE  ID: 

SUBJ0044 

DESCRIPTION: 

DO  YOU  AGREE: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS,  N12 

UNDERSTAND  MOST  OF  MATH  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 

AGREE 

UNDECIDED 

disagree 

STRONGLY  DISAGREE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


CONCEPTS  ARE  AS  IMPORTANT  AS  OPER 


NAEP  ID: 


M810705 


TYPE  OF  CONTRAST: 

CLASS 

001 

M810705A 

(01  ) 

00000 

002 

M810705B 

(02  ) 

10000 

003 

M810705C 

(03  ] 

1 01000 

004 

M810705D 

(04  ) 

00100 

005 

M810705E 

(05  ] 

1 00010 

006 

M810705M 

(M  ) 

1 00001 

CONDITIONING 

VARIABLE  ID: 

SUBJ0( 

DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


NOS,  SOS,  N12 


NAEP  ID: 

M810706 

TYPE  OF  CONTRAST: 

CLASS 

001  M810706A 

(01  ) 

1 00000 

002  M810706B 

(02  ] 

1 10000 

003  M810706C 

(03  ] 

) 01000 

004  M810706D 

(04  ] 

) 00100 

005  M810706E 

(05  ] 

1 00010 

006  M810706M 

(M  ] 

1 00001 

CONDITIONING 

VARIABLE  ID: 

SUBJ0046 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


DESCRIPTION: 


agree /DISAGREE:  ALL  CAN  DO  WELL  IN  MATH  IF  TRY 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 

001  M810711A  (01 

002  M810711B  (02 

003  M810711C  (03 

004  M810711D  (04 

005  M810711E  (05 

006  M810711M  (M 


N08,  S08,  N12 

M810711 

CLASS 

) 00000 
) 10000 
) 01000 
) 00100 
) 00010 
) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

STRONGLY  AGREE 
AGREE 
UNDECIDED 
DISAGREE 

STRONGLY  DISAGREE 
MISSING 


CONDITIONING  VARIABLE  ID;  BACK0082 
DESCRIPTION:  SINCE  KDG, 

grades /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL; 

NAEP  ID:  B008301 

TYPE  OF  CONTRAST:  CLASS 


GRADES  ATTENDED  IN  THIS  STATE? 


001  B008301A  (01 

002  B008301B  (02 

003  B008301C  (03 

004  B008301D  (04 

005  B008301E  (05 

006  B008301M  (M 


) 00000 
) 10000 
) 01000 
) 00100 
) 00010 
) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

Less  than  i grade 

1-2  GRADES 
3-5  GRADES 
6-9  GRADES 
MORE  THAN  9 GRADES 

missing 


CONDITIONING  VARIABLE  ID:  BACK0083 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


DESCRIBE  YOUR  OVERALL  GRADES  SINCE  9TH  GRADE 
N12 


001 

002 

003 

004 

005 

006 
007 


B009901A  (01 
B009901B  (02 
B009901C  (03 
B009901D  (04 
B009901E  (05 
B009901F  (06 
B009901M  (M 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


B009901 

CLASS 

) 000000 
) 100000 
) 010000 
) 001000 
) 000100 
) 000010 
) 000001 

BACK0084 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MOSTLY  A'S 
MOSTLY  B'S 
MOSTLY  C ' S 
MOSTLY  D'S 
MOSTLY  BELOW  D'S 
CLASSES  NOT  GRADED 
MISSING 


WHICH  BEST  DESCRIBES  YOUR  HIGH  SCHOOL  PROGRAM 
N12 


001  GENERAL 

002  ACADEMIC 

003  VOC/TECH 

004  OTHER 

005  missing 


(01 

(02 

(03 

(04 

(M 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 


B008501 

CLASS' 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 

BACK0085 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

GENERAL 

ACADEMIC/COLLEGE  PRE 
VOCATIONAL  OR  TECHN 
OTHERRADES 

missing 


WHAT  WILL  TAKE  LARGEST  AMT. 
N12 

B005501 


OF  TIME  AFTER  HIGH-SCH 


TYPE  OF  CONTRAST: 

CLASS 

001  WORKING 

(01 

) 000000 

002  VOC  COLL 

(02 

) 100000 

003  2 YR  COL 

(03 

) 010000 

004  4 YR  COL 

(04 

) 001000 

005  MILITARY 

(05 

) 000100 

006  OTHER 

(06  ; 

I 000010 

007  MISSING 

(M  ; 

I 000001 

CONDITIONING 

VARIABLE  ID: 

BACK0086 

DESCRIPTION: 

KIND  OF  WORK  DONE 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

MOTHOCC 

TYPE  OF  CONTRAST: 

CLASS 

001  NOT  PRE 

(01. M ) 

1 0000000000000000 

002  OFFICE  W 

(02  ) 

1 1000000000000000 

003  TECHNIC 

(03  ] 

1 0100000000000000 

004  PROT  SRV 

(04  ] 

1 0010000000000000 

005  SALES 

(05  ) 

' 0001000000000000 

006  OWNER 

(06  ) 

' 0000100000000000 

007  SERV  WRK 

(07  ) 

' 0000010000000000 

008  SKILL  T 

(08  ) 

1 0000001000000000 

009  LABORER 

(09  ) 

0000000100000000 

010  OPERATOR 

(10  ) 

0000000010000000 

Oil  FARMER 

(11  ) 

0000000001000000 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

WORKING  FULL-TIME 
VOCATIONAL  COLLEGE 
ATTEND  2 YR  COLLEGE 
ATTEND  4 YEAR  COLLEG 
MILITARY  SERVICE 
OTHERS 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  PRESENT  IN  HOUSEHOLD 
OFFICE  WORKER 
TECHNICAL 

PROTECTIVE  SERVICE 

SALES 

OWNER 

SERVICE  WORKER 
SKILLED  TRADESPERSON 
LABORER 
OPERATOR 

FARMER/FARM  MANAGER 


17 

16 


0"'( 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


ERIC 


012  HOMEMAKE  (12 

013  MANAGER  (13 

014  MILITARY  (14 

015  TEACHER 

016  PROFESSl 

017  PROFESS2 


(15 

(16 

(17 


) 0000000000100000 
) 0000000000010000 
) 0000000000001000 
) 0000000000000100 
) 0000000000000010 
) 0000000000000001 


HOMEMAKER 
MANAGER 
MILITARY 
SCHOOL  TEACHER 
PROFESSIONAL  1 
PROFESSIONAL  2 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 

CONDITIONING  VAR  LABEL; 

NAEP  ID;  BOllAOl 

TYPE  OF  CONTRAST;  CLASS 

001  BOllAOlM  (M  ) 0 

002  BOllAOlN  (0  ) 1 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES/ ASSESSMENTS ; 

CONDITIONING  VAR  LABEL; 

NAEP  ID;  FATHOCC 

TYPE  OF  CONTRAST;  CLASS 


BACK0087 

KIND  OF  WORK  DONE  BY  YOUR  MOTHER/ FEMALE  GUARDIAN? 
N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

MISSING 

NOT  PRESENT  IN  HOUSEHOLD 


BACK0088 

KIND  OF  WORK  DONE  BY  YOUR  FATHER/MALE  GUARDIAN? 
N12 


001 

NOT  PRES 

(01, M 

) 0000000000000000 

002 

OFFICE 

(02 

) 1000000000000000 

003 

TECHNIC 

(03 

) 0100000000000000 

004 

PROT  SRV 

(04 

) 0010000000000000 

005 

SALES 

(05 

) 0001000000000000 

006 

OWNER 

(06 

) 0000100000000000 

007 

SERV  WRK 

(07 

) 0000010000000000 

008 

SKILL  TR 

(08 

) 0000001000000000 

009 

LABORER 

(09 

) 0000000100000000 

010 

OPERATOR 

(10 

) 0000000010000000 

Oil 

FARMER 

(11 

) 0000000001000000 

012 

HOMEMAKE 

(12 

) 0000000000100000 

013 

MANAGER 

(13 

) 0000000000010000 

014 

MILITARY 

(14 

) 0000000000001000 

015 

TEACHER 

(15 

) 0000000000000100 

016 

PROFESSl 

(16 

) 0000000000000010 

017 

PROFESS2 

(17 

) 0000000000000001 

CONDITIONING 

VARIABLE 

ID;  BACK0089 

DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


N12 


B012A02 

CLASS 


001  B012A02M  (M  ) 

0 

002  B012A02N  (0  ) 

1 

CONDITIONING  VARIABLE  ID; 

SUBJ0047 

DESCRIPTION; 

ARE  YOU  TAKING 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

N12 

NAEP  ID; 

M810901 

TYPE  OF  CONTRAST; 

CLASS 

001  M810901Y  (01  ) 

00 

002  M810901N  (02  ) 

10 

003  M810901M  (M  ) 

01 

CONDITIONING  VARIABLE  ID; 

SUBJ0048 

DESCRIPTION; 

WHAT  GRADE  DID 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

N12 

NAEP  ID; 

M811801 

TYPE  OF  CONTRAST; 

CLASS 

001  M811801A  (01 

) 00000 

002  M811801B  (02 

) 10000 

003  M811801C  (03 

) 01000 

004  M811801D  (04 

) 00100 

005  M811801E  (05 

) 00010 

006  M811801M  (M 

) 00001 

CONDITIONING  VARIABLE  ID; 

SUBJ0049 

DESCRIPTION; 

HOW  LONG  HAVE 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

N12 

NAEP  ID; 

M811001 

TYPE  OF  CONTRAST; 

CLASS 

001  M811001A  (01 

) 0000 

002  M811001B  (02 

) 1000 

003  M811001C  (03 

) 0100 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

NOT  PRESENT  IN  HOUSEHOLD 
OFFICE  WORKER 
TECHNICAL 

PROTECTIVE  SERVICES 

SALES 

OWNER 

SERVICE  WORKER 
SKILLED  TRADEPERSON 
LABORER 
OPERATOR 

FARMER/ FARM  MANAGER 

HOMEMAKER 

MANAGER 

MILITARY 

SCHOOL  TEACHER 

PROFESSIONAL  1 

PROFESSIONAL  2 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

MISSING 

NOT  PRESENT  IN  HOUSEHOLD 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

NO 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

HAVNT.  TAKEN  1ST  YEAR 

BEFORE  9TH  GRADE 

9TH  GRADE 

lOTH  GRADE 

IITH  OR  12TH  GRADE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


17 

16 


MORE  THAN  1 ^”EAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
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Table  C-4  (continued) 
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004  M811001D  (04  ) 0010 

005  M811001M  (M  ) 0001 


NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0050 

HOW  LONG  HAVE  YOU  TAKEN  BUSINESS  OR  CONSUMER  MATH 


M811002 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811002A  (01  ) 0000 

002  M811002B  (02  ) 1000 

003  M811002C  (03  ) 0100 

004  M811002D  (04  ) 0010 

005  M811002M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0051 

HOW  long  taken  applied  MATH? 
N12 

M811014 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811014A  (01  ) 0000 

002  M811014B  (02  ) 1000 

003  M811014C  (03  ) 0100 

004  M811014D  (04  ) 0010 

005  M811014M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0052 

HOW  LONG  HAVE  YOU  TAKEN  INTRO  (PRE)  - ALGEBRA 
N12 


M811003 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811014A  (01  ) 0000 

002  M811014B  (02  ) 1000 

003  M811014C  (03  ) 0100 

004  M811014D  (04  ) 0010 

005  M811014M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0053 

HOW  LONG  HAVE  YOU  STUDIED  FIRST-YEAR  ALGEBRA  ■ 
N12 


M811004 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811004A  (01  ) 0000 

002  M811004B  (02  ) 1000 

003  M811004C  (03  ) 0100 

004  M811004D  (04  ) 0010 

005  M811004M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SUBJ0054 

HOW  long  have  you  taken  geometry 
N12 

M811005 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  M811005A  (01  ) 0000 

002  M811005B  (02  ) 1000 

003  M811005C  (03  ) 0100 

004  M811005D  (04  ) 0010 

005  M811005M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
NOT  STUDIED 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0055 

HOW  long  HAVE  YOU  TAKEN  SECOND-YEAR  ALGEBRA 
N12 


M811006 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  M811006A  (01  ) 0000 

002  M811006B  (02  ) 1000 

003  M811006C  (03  ) 0100 

004  M811006D  (04  ) 0010 

005  M811006M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0056 

HOW  long  HAVE  YOU  TAKEN  TRIGONOMETRY 
N12 


M811007 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


O 


4Q8 
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001  M811007A  (01  ) 0000 

002  M811007B  (02  ) 1000 

003  M811007C  (03  ) 0100 

004  M811007D  (04  ) 0010 

005  M811007M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  year 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0057 

HOW  LONG  HAVE  YOU  TAKEN  PRE-CALC  (3RD  YR  ALGEBRA) 

N12 

M811008  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

rriv  oo  NUMBER  OF  INDEPENDENT  CONTRASTS  ; 


001  M811008A  (01  ) 0000 

002  M811008B  (02  ) 1000 

003  M811008C  (03  ) 0100 

004  M811008D  (04  ) 0010 

005  M811008M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0058 

HOW  LONG  TAKEN  UNIFIED/INTEGRTD/SEQ  MATH? 

N12 

M811012  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

class  NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  M811012A  (01  ) 0000 

002  M811012B  (02  ) 1000 

003  M811012C  (03  ) 0100 

004  M811012D  (04  ) 0010 

.005  M811012M  (M  . ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS ;' 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SUBJ0059 

HOW  LONG  HAVE  YOU  TAKEN  PROBABILITY  OR  STATISTICS 
N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


M811009 

CLASS 


001  M811009A  (01  ) 0000 

002  M811009B  (02  ) 1000 

003  M811009C  (03  ) 0100 

004  M811009D  (04  ) 0010 

005  M811009M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2  YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID;  SUBJ0060 

DESCRIPTION;  HOW  LONG  HAVE  YOU  TAKEN  CALCULUS 


GRADES /ASSESSMENTS;  N12 

CONDITIONING  VAR  LABEL; 

NAEP  ID:-  M811011 

TYPE  OF  CONTRAST;  CLASS 

001  M811011A  (01  ) 0000 

002  M811011B  (02  ) 1000 

003  M811011C  (03  ) 0100 

004  M811011D  (04  ) 0010 

005  M811011M  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MORE  THAN  1 YEAR 
1 SCHOOL  YEAR 
1/2- YEAR  OR  LESS 
NOT  STUDIED 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0061 

IN/HAVE  TAKEN  MATH  ADVANCED  PLACEMENT  CLASS? 

N12 

M812801  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CTjvss  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  M812801Y  (01  ) 00 

002  M812801N  (02  ) 10 

003  M812801M  (M  ) 01 


YES 

NOCE  OR  TWICE  A WEEK 
MISSING 


CONDITIONING  VARIABLE  ID;  SCHL0003 
DESCRIPTION;  BEST  DESCRIBES  HOW 

GRADES/ASSESSMENTS;  N04,  S04 

CONDITIONING  VAR  LABEL; 

NAEP  ID:  C030901 

TYPE  OF  CONTRAST;  CLASS 


4TH  GR  ARE  ORGANIZED? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C030901A  (01  ) 000 

002  C030901B  (02  ) 100 

003  C030901C  (03  ) 010 

004  C030901M  (M  ) 001 


SELF  CONTAINED 
DEPARTMENTALIZED 
REGROUPED 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 


SCHL0004 

4TH  GRADERS  ASSIGNED  BY  ABILITY/ACHIEVEMENT  LEVEL? 

N04,  S04 

C037101  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


5 

4 


5 

4 


5 

4 


5 

4 


4 

3 


5 


550 


499 


REST  COPY  AVAILABLE 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT 

001  C037101A 

(01  ) 

0000 

YES, 

BY  MATH 

ABILITY 

002  C037101B 

(02  ) 

1000 

YES, 

READING 

ABILITY 

003  C037101C 

(03  ) 

0100 

YES, 

GENERAL 

ABILITY 

004  C037101N 

(04  ) 

0010 

NO 

005  C037101M 

(M  ) 

0001 

MISSING 

CONDITIONING 

VARIABLE  ID: 

SCHL0005 

DESCRIPTION: 

HOW  OFTEN  IS 

4 TH -GRADER  INSTRUCTED  IN 

MATH? 

GRADES /ASSESSMENTS:  N04,  S04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C031212 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C031212A 

(01 

) 00000 

002 

C031212B 

(02 

) 10000 

003 

C031212C 

(03 

) 01000 

004 

C031212D 

(04 

) 00100 

005 

C031212E 

(05 

) 00010 

006 

C031212M 

(M 

) 00001 

EVERY  DAY 

THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  C031205A  (01  ) 00000 

002  C031205B  (02  ) 10000 

003  C031205C  (03  ) 01000 

004  C031205D  (04  ) 00100 

005  C031205E  (05  ) 00010 

006  C031205M  (M  ) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

EVERY  DAY 

THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


ID:  SCHL0006 

HOW  OFTEN  IS  4TH-GRADER  INSTRUCTED  IN  SCIENCE? 
N04,  S04 

C031205 
CLASS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0007 

HOW  OFTEN  IS  4TH-GRADER  INSTRUCTED  IN  READING? 

N04,  S04 

C031213  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C031213A 

(01 

) 00000 

002 

C031213B 

(02 

) 10000 

003 

C031213C 

(03 

) 01000 

004 

C031213D 

(04 

) 00100 

005 

C031213E 

(05 

) 00010 

006 

C031213M 

(M 

) 00001 

EVERY  DAY 

THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0008 

HOW  OFTEN  IS  4TH-GRADER  INSTRUCTED  IN  ARTS? 

N04,  S04 

C031214  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C031214A 

(01 

) 00000 

002 

C031214B 

(02 

) 10000 

003 

C031214C 

(03 

) 01000 

004 

C031214D 

(04 

) 00100 

005 

C031214E 

(05 

) 00010 

006 

C031214M 

(M 

) 00001 

EVERY  DAY 

THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0009 

HAS  MATH  BEEN  IDENTIFIED  AS  A PRIORITY? 

N04,  S04,  NOS,  SOS 

CO 3 1603  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C031603Y 

(01 

) 00 

002 

C031603N 

(02 

) 10 

003 

C031603M 

(M 

) 01 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOOlO 

HAS  SCIENCE  BEEN  IDENTIFIED  AS  A PRIORITY? 

N04,  S04,  NOS,  SOS 

C031607  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C031607Y 

(01 

) 00 

002 

C031607N 

(02 

) 10 

003 

C031607M 

(M 

) 01 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 


SCHLOOll 


4 


6 

5 


6 

5 


6 

5 


6 

5 


3 

2 


3 

2 


o "00 

ERIC 


551 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


DESCRIPTION: 

GRADES /ASSESSMENTS: 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C031601Y  (01 

002  C031601N  (02 

003  C031601M  (M 


HAS  READING  BEEN  IDENTIFIED  AS  A PRIORITY? 
N04,  S04 


C031601 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0012 


DESCRIPTION: 
GRADES/ASSESSMENTS : ■ 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C031610Y  (01 

002  C031610N  (02 

003  C031610M  (M 


HAS  ARTS  BEEN  IDENTIFIED  AS  A PRIORITY? 
N04,  S04,  NOS,  SOS 


C031610 

CLASS 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0013 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C031606Y  (01 

002  C031606N  (02 

003  C031606M  (M 


HAS  SUBJECT  INTEGRATION  BEEN  A PRIORITY? 
N04,  S04,  NOS,  SOS 


C031606 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0014 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C035701Y  (01 

002  C035701N  (02 

003  C035701M  (M 


COMPUTERS  AVAILABLE  ALL  THE  TIME  IN  CLASSROOM? 
N04,  S04,  NOS,  SOS,  N12 


C035701 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0015 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C035702Y  (01 

002  C035702N  (02 

003  C035702M  (M 


COMPUTERS  GROUPED  IN  SEPARATE  LAB  AND  AVAILABLE? 
N04,  S04,  NOS,  SOS,  N12 


C035702 

CLASS 

00 

10 

01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0016 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


COMPUTERS  AVAILABLE  TO  BRING  TO  ROOM  WHEN  NEEDED? 
N04,  S04,  NOS,  SOS,  N12 


C035703 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C035703Y  (01 

002  C035703N  (02 

003  C035703M  (M 


) 00 
) 10 
) 01 


YES 

NO 

MISSING 


3 

2 


3 

2 


3 

2 


3 

2 


O 

ERIC 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES / ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0017 

SCHOOL  W/  SPECIAL  FOCUS  ON  MATH? 

N04,  S04,  NOS,  SOS,  N12 

C037201  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  C037201Y  (01  ) 0 

002  C037201M  (M  ) 1 


YES 

MISSING 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOOIS 

SCHOOL  W/  SPECIAL  FOCUS  ON  SCIENCE? 

N04,  S04,  NOS,  SOS,  N12 

C037202  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037202Y  (01  ) 0 

002  C037202M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


SCHL0019 

SCHOOL  W/  SPECIAL  FOCUS  ON  MATH? 


BEST  COPY  AVAILABLE 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


N04,  S04 

C037207 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037207Y  (01  ) 0 

002  C037207M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0020 

SCHOOL  W/  SPECIAL  FOCUS  ON  ARTS? 

N04,  S04,  NOS,  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


C037204 

CLASS 


001  C037204Y  (01  ) 0 

002  C037204M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0021 

SCHOOL  W/  SPECIAL  FOCUS  ON  OTHER? 

N04,  S04,  NOS,  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


C037205 

CLASS 


001  C037205Y  (01  ) 0 YES 

002  C037205M  (M  ) 1 MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0022 

DESCRIPTION:  SCHOOL  NOT  A SPECIAL  FOCUS  SCHOOL? 

GRADES /ASSESSMENTS:  N04,  S04,  NOS,  SOS,  Nl2 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C037206 

TYPE  OF  CONTRAST:  CLASS 

001  C037206Y  (01  ) 0 

002  C037206M  (M  ) 1 

SCHL0023 

SCHOOL  FOLLOW  DISTRICT/ STATE  MATH  CURRICULUM? 

N04,  S04,  NOS,  SOS,  N12 

C037301  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037301Y  (01  ) 0 

002  C037301M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


037206 

LASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0024 

SCHOOL  FOLLOW  DISTRICT/STATE  SCIENCE  CURRICULUM? 

N04,  S04,  N08,  SOS,  N12 

C037302  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037302Y  (01  ) 0 

002  C037302M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 


SCHL0025 

SCHOOL  FOLLOW  DISTRiCT/STATE  READING  CURRICULUM? 

N04,  S04 

C037303  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037303Y  (01  ) 0 

002  C037303M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0026 

SCHOOL  FOLLOW  DISTRiCT/STATE  ARTS  CURRICULUM? 

N04,  S04,  NOS,  SOS.  N12 

C037304  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037304Y  (01  ) 0 

002  C037304M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0027 

SCHOOL  FOLLOW  DISTRICT/STATE  FOR  NONE  OF  ABOVE? 

N04,  S04,  NOS,  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


C037305 

CLASS 


001  C037305Y  (01  ) 0 

002  C037305M  (M  ) 1 


YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


0"2 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 

SCHL0028 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  MATH? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C037401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C037401Y-  (01  ) 

002  C037401M  (M  ) 

0 yes 

1 MISSING 

CONDITIONING  VARIABLE  ID: 

DESCRIPTION: 

grades /ASSESSMENTS : 

SCHL0029 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  SCIENCE? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

PQ37402  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

^LAgg  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C037402Y  (01  ) 

002  C037402M  (M  ) 

0 yes 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSES  SMENTS : 

SCHL0030 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  READING? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

P037403  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

^^gg  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C037403Y  (01  ) 

002  C037403M  (M  ) 

0 yes 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 

SCHL0031 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  ARTS? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

P037404  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

^LASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C037404Y  (01  ) 

002  C037404M  (M  ) 

, 0 yes 

, 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0032 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  OTHER? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C037405  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

^LAgg  NUMBER  OF  INDEPENDENT  CONTRASTS : 1 

001  C037405Y  (01 

002  C037405M  (M 

) 0 yes 

j 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0033  

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  NONE  ABOVE? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

PQ37406  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C037406Y  (01 

002  C037406M  (M 

) 0 yes  - 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0034 

4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  MATH? 
N04,  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C037501  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

^jj^gg  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C037501Y  (01 

002  C037501M  (M 

) 0 yes 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 

SCHL0035 

4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  SCIENCE? 

N04.  S04 

C037502  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C037502Y  (01 

002  C037502M  (M 

) 0 yes 

) 1 MISSING 

UCbL.Kllr'TlUIM  : 

GRADES /ASSESSMENTS : 

N04.  S04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

P037503  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

^jj^gg  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

O 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


001  C037503Y  (01  ) 0 

002  C037503M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0037 

4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  ARTS’ 
N04,  S04 


C037504 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037504Y  (01  ) 0 

002  C037504M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0038 

4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  NONE  OF  ABOVE’ 
N04,  S04 


C037505 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037505Y  (01  ) 0 

002  C037505M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0039 

4TH  GRADERS  IN  SUMMER  PROGRAMS 
N04,  S04 

C037601 

CLASS 


I MATH? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037601Y  (01  ) 0 

002  C037601M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0040 

4TH  GRADERS  IN  SUMMER  PROGRAMS 
N04,  S04 

C037602 

CLASS 


I SCIENCE? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  C037602Y  (01  ) 0 

002  C037602M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0041 

4TH  GRADERS  IN  SUMMER  PROGRAMS 
N04,  S04 

C037603 

CLASS 


J READING? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037603Y  (01  ) 0 

002  C037603M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0042 

4TH  GRADERS  IN  SUMMER  PROGRAMS 
N04,  S04 

C037604 

CLASS 


I ARTS? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037604Y  (01  ) 0 

002  C037604M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0043 

4TH  GRADERS  IN  SUMMER  PROGRAMS  IN  NONE  OF  ABOVE’ 
N04,  S04 


C037605 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037605Y  (01  ) 0 

002  C037605M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


SCHL0044 

WHICH  BEST  DESCRIBES  PRIMARY  WA 
N04,  S04,  N08,  S08,  N12 

C036601 


TYPE  OF  CONTRAST: 

CLASS 

001 

C036601A 

(01 

) 0000 

002 

C036601B 

(02 

) 1000 

003 

C036601C 

(03 

) 0100 

004 

C036601D 

(04 

) 0010 

005 

C036601M 

(M 

) 0001 

LIBRARY  STAFFED? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NO  LIBRARY  IN  SCHOOL 
LiBRARY-NO/VOL  STAFF 
PART-TIME  STAFF 
FULL-TIME  STAFF 
MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


5 

4 


^<04 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0045 

INVOLVE  PARENTS  AS  AIDES  IN  CLASSROOM? 
N04,  S04,  NOS,  SOS,  N12 


C032207 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032207A  (01  ) 000 

002  C032207B  (02  ) 100 

003  C032207N  (03  ) 010 

004  C032207M  (M  ) 001 


YES,  ROUTINELY 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0046 

HAVE  PARENTS  REVIEW/ SIGN  HOMEWORK? 
N04,  S04,  NOS,  SOS,  N12 


C032209 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032209A  (01  ) 000 

002  C032209B  (02  ) 100 

003  C032209N  (03  ) 010 

004  C032209M  (M  ) 001 


YES,  ROUTINELY 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0047 

ASSIGN  HOMEWORK  STUDENTS  DO  WITH  PARENTS? 
N04,  S04,  NOS,  SOS,  N12 


C032210 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032210A  (01  ) 000 

002  C032210B  (02  ) 100 

003  C032210N  (03  ) 010 

004  C032210M  (M  ) 001 


YES,  ROUTINELY 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL004S 

HAVE  A PTUIENT  VOLUNTEER  PROGRAM? 
N04,  S04,  NOS,  SOS,  N12 

C032211 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032211A  (01  ) 000 

002  C032211B  (02  ) 100 

003  C032211N  (03  ) 010 

004  C032211M  (M  ) 001 


YES,  ROUTINELY* 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0049 

WHAT  % OF  "PARENTS  IN  PARENT -TEACHER  ORGS? 
N04,  S04,  NOS,  SOS,  N12 


C037701 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037701A  (01  ) 0000 

002  C037701B  (02  ) 1000 

003  C037701C  (03  ) 0100 

004  C037701D  (04  . ) 0010 

005  C037701M  (M  ) 0001 


0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


WHAT  % OF  PARENTS  IN  OPEN  HOUSE/BACK  SCHOOL  NIGHT? 
N04,  S04,  NOS,  SOS,  N12 


C037702 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037702A  (01  ) 0000 

002  C037702B  (02  ) 1000 

003  C037702C  (03  ) 0100 

004  C037702D  (04  ) 0010 

005  C037702M  (M  ) 0001 


0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


4 

3 


4 

3 


4 

3 


4 

3 


5 

4 


5 

4 


ONDITIONING  VARIABLE  ID: 
ESCRIPTION: 

RADES/ ASSESSMENTS : 
ONDITIONING  VAR  LABEL: 
AEP  ID: 

YPE  OF  CONTRAST: 


SCHL0051 

WHAT  % OF  PARENTS  IN  PARENT-TEACHER 
N04,  S04,  NOS,  SOS,  N12 

C037703 
CLASS 


001  C037703A  (01  ) 0000 

002  C037703B  (02  ) 1000 

003  C037703C  (03  ) 0100 

004  C037703D  (04  ) 0010 

005  C037703M  (M  ) 0001 


CONFERENCES? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

0-25% 

26-50% 

51-75% 

76-100% 

MISSING 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0052 

WHAT  % PARENTS  INVOLVED  MAKING  CURRICULUM  DECISION 
N04,  S04,  N08,  S08,  N12 


C037704 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037704A  (01  ) 0000 

002  C037704B  (02  ) 1000 

003  C037704C  (03  ) 0100 

004  C037704D  (04  ) 0010 

005  C037704M  (M  ) 0001 


0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0053 

WHAT  % OF  PARENTS  IN  VOLUNTEER  PROGRAMS? 
N04,  S04,  N08,  S08,  N12 


C037705 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037705A  (01  ) 0000 

002  C037705B  (02  ) 1000 

003  C037705C  (03  ) 0100 

004  C037705D  (04  ) 0010 

005  C037705M  (M  ) 0001 


0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0054 

IS  STUDENT  ABSENTEEISM  A PROBLEM  IN  YOUR  SCHOOL’ 
N04,  S04,  N08,  S08,  N12 


C032402 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032402A  (01  ) 0000 

002  C032402B  (02  ) 1000 

003  C032402C  (03  ) 0100 

004  C032402D  (04  ) 0010 

005  C032402M  (M  ) 0001 


SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0055 

IS  STUDENT  TARDINESS  A PROBLEM  IN  YOUR  SCHOOL’ 
N04,  S04,  N08,  S08,  N12 


C032401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032401A  (01  ) 0000 

002  C032401B  (02  ) 1000 

003  C032401C  (03  ) 0100 

004  C032401D  (04  ) 0010 

005  C032401M  (M  ) 0001 


SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0056 

ARE  PHYSICAL  CONFLICTS  A PROBLEM  IN  YOUR  SCHOOL’ 
N04,  S04,  N08,  S08,  N12 


C032404 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  C032404A  (01  ) 0000 

002  C032404B  (02  ) 1000 

003  C032404C  (03  ) 0100 

004  C032404D  (04  ) 0010 

005  C032404M  (M  j 0001 


SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0057 

IS  TEACHER  ABSENTEEISM  A PROBLEM  IN  YOUR  SCHOOL’ 
N04,  S04.  N08,  S08,  N12 


C032406 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032406A  (01  ) 0000 

002  C032406B  (02  ) 1000 

003  C032406C  (03  ) 0100 

004  C032406D  (04  ) 0010 

005  C032406M  (M  ) 0001 


SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0058 

ARE  RACE/CULT.  CONFLICTS  A PROBLEM  IN  YOUR  SCHOOL’ 
N04,  S04,  N08,  S08.  N12 


C032407 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032407A  (01  ) 0000 

002  C032407B  (02  ) 1000 


SERIOUS 

MODERATE 
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003  C032407C  (03 

004  C032407D  (04 

005  C032407M  (M 


) 0100 
) 0010 
) 0001 


MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0059 

DESCRIPTION:  IS  STUDENT  HEALTH  A PROBLEM  IN  YOUR  SCHOOL? 

GRADES/ASSESSMENTS:  N04,  S04,  NOS,  SOS,  N12 


CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C03240S 

CLASS 

001  C03240SA 

(01 

) 0000 

002  C03240SB 

(02 

) 1000 

003  C03240SC 

(03 

) 0100 

004  C03240SD 

(04 

) 0010 

005  C03240SM 

(M 

) 0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRI PTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0060 

IS  LACK  OF  PARENT  INVLMT  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  S04,  NOS,  SOS,  N12 

C032409  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001 

C032409A 

(01  , 

) 0000 

002 

C032409B 

(02 

) 1000 

003 

C032409C 

(03 

) 0100 

004 

C032409D 

(04 

) 0010 

005 

C032409M 

(M 

) 0001 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0061 

IS  STUD  USE  OF  ALCOHOL  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  S04,  NOS,  SOS,  N12 

C032410  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS : 


001 

C032410A 

(01 

) 0000 

002 

C032410B 

(02 

) 1000 

003 

C032410C 

(03 

) 0100 

004 

C032410D 

(04 

) 0010 

005 

C032410M 

(M 

) 0001 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0062 

IS  STUDENT  TOBACCO  USE  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  S04,  NOS,  SOS,  N12 

C032411  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C032411A 

(01 

) 0000 

002 

C032411B 

(02 

) 1000 

003 

C032411C 

(03 

) 0100 

004 

C032411D 

(04 

) 0010 

005 

C032411M 

(M 

) 0001 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


5 

4 


5 

4 


5 

4 


5 

4 


CONDITIONING  VARIABLE  ID:  SCHL0063 

DESCRIPTION:  IS  STUDENT  DRUG  USE  A PROBLEM  IN  YOUR  SCHOOL? 

GRADES /ASSESSMENTS : N04,  S04,  NOS,  SOS,  N12 


CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

C032412 

TYPE  OF  CONTRAST: 

CLASS 

001  C032412A 

(01 

) 0000 

002  C032412B 

(02 

) 1000 

003  C032412C 

(03 

) 0100 

004  C032412D 

(04 

) 0010 

005  C032412M 

(M 

) 0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


5 

4 


CONDITIONING  VARIABLE  ID:  SCHL0064 

DESCRIPTION;  ARE  GANG  ACTIVITIES  A PROBLEM  IN  YOUR  SCHOOL? 

GRADES /ASSESSMENTS:  N04,  S04,  NOS,  SOS,  N12 


CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C032413 

CLASS 

001  C032413A 

(01 

) 0000 

002  C032413B 

(02 

) 1000 

003  C032413C 

(03 

) 0100 

004  C032413D 

(04 

) 0010 

005  C032413M 

(M 

) 0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


5 

4 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


SCHL0065 

IS  STUDENT  MISBEHAVIOR  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  S04,  NOS,  SOS,  N12 

C032414  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


5 
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TYPE  OF  CONTRAST:  CLASS 

001  C032414A  (01  ) 0000 

002  C032414B  (02  ) 1000 

003  C032414C  (03  ) 0100 

004  C032414D  (04  ) 0010 

005  C032414M  (M  ) 0001 


NUMBER  OF  INDEPENDENT  CONTRASTS 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0066 

IS  STUDENT  CHEATING  A PROBLEM  IN  YOUR  SCHOOL? 
N04,  S04,  NOS,  SOS,  N12 


C032415 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032415A  (01  ) 0000 

002  C032415B  (02  ) 1000 

003  C032415C  (03  ) 0100 

004  C032415D  (04  ) 0010 

005  C032415M  (M  ) 0001 


SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0067  * 

IS  TEACHER  MORALE  POS.  OR  NEG.? 
N04,  S04,  NOS,  SOS,  N12 


C032502 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032502A  (01  ) 0000 

002  C032502B  (02  ) 1000 

003  C032502C  (03  ) 0100 

004  C032502D  (04  ) 0010 

005  C032502M  (M  ) 0001 


VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL006S 

ARE  STUDENT  ATTITUDES  TO  ACADEMICS  POS.  OR  NEG.? 
N04,  S04,  NOS,  SOS,  N12 


C032503 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032503A  (01  ) 0000 

002  C032503B  (02  ) 1000 

003  C032503C  (03  ) 0100 

004  C032503D  (04  ) 0010 

005  C032503M  (M  ) 0001 


VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0069 

IS  PARENT  SUPPORT  FOR  ACHIEVEMENT  POS.  OR  NEG.? 
N04,  S04,  NOS,  SOS,  N12 


C032505 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032505A  (01  ) 0000 

002  C032505B  (02  ) 1000 

003  C032505C  (03  ) 0100 

004  C032505D  (04  ) 0010 

005  C032505M  (M  ) 0001 


VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0070 

IS  REGARD  FOR  SCHOOL  PROPERTY  POS.  OR  NEG.? 

N04,  S04,  NOS,  SOS,  N12 

C032506  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032506A  (01  ) 0000 

002  C032506B  (02  ) 1000 

003  C032506C  (03  ) 0100 

004  C032506D  (04  ) 0010 

005  C032506M  (M  ) 0001 


VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C033601A  (01 

002  C033601B  (02 

003  C033601C  (03 

004  C033601D  (04 

005  C033601M  (M 


SCHL0071 

% ABSENT  ON  AVERAGE  DAY? 
N04,  S04,  NOS,  SOS,  N12 

C033601 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


CONDITIONING  VARIABLE  ID:  SCHL0072 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

0-2% 

3-5% 

6-10% 

MORE  THAN  10% 

MISSING 


4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


O '08 

ERLC 
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DESCRIPTION: 

WHAT  % OF  TEACHERS  ABSENT  ON 

GIVEN  DAY? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C036501 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036501A  (01  ) 

0000 

0-2% 

002  C036501B  (02  ) 

1000  • 

3-5% 

003  C036501C  (03  ) 

0100 

6-10% 

004  C036501D  (04  ) 

0010 

MORE  THAN  10% 

005  C036501M  (M  ) 

0001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0073 

DESCRIPTION: 

% OF  STUDS  EROLLED  AT  START  OF  YR  EROLLED  AT  END? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

NAEP  ID: 

C037801 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037801A  (01  ) 

00000000 

98-100% 

002  C037801B  (02  ) 

10000000 

95-97% 

003  C037801C  (03  ) 

01000000 

90-94% 

004  C037801D  (04  ) 

00100000 

80-89% 

005  C037801E  (05  ) 

00010000 

70-79% 

006  C037801F  (06  ) 

1 00001000 

60-69% 

007  C037801G  (07  ) 

1 00000100 

50-59% 

008  C037801H  (08  ) 

1 00000010 

LESS  THAN  50% 

009  C037801M  (M  ) 

1 00000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0074 

REPEATING  4TH  GRADE? 

DESCRIPTION: 

% OF  4TH  GRADERS  HELD  BACK  & 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

NAEP  ID: 

C037901 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037901A  (01  ; 

1 00000 

0% 

002  C037901B  (02  ! 

1 10000 

1-2% 

003  C037901C  (03  ! 

1 01000 

3-5% 

004  C037901D  (04  ! 

1 00100 

6-10% 

005  C037901E  (05 

> 00010 

MORE  THAN  10% 

006  C037901M  (M 

> 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0075 

DESCRIPTION: 

% OF  FULL  TIME  TEACHERS  LEFT 

BEFORE  END  OF  YR? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08,  S08,  Nl2 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

NAEP  ID: 

C038001 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C038001A  (01 

) 0000000 

0% 

002  C038001B  (02 

) 1000000 

1-2% 

003  C038001C  (03 

) 0100000 

3-5% 

004  C038001D  (04 

) OOlO'OOO 

6-10% 

005  C038001E  (05 

) 0001000 

11-15% 

006  C038001F  (06 

) 0000100 

16-20% 

007  C038001G  (07 

) 0000010 

MORE  THAN  20% 

008  C038001M  (M 

) 0000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0076 

DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C038301Y  (01 

002  C038301N  (02 

003  C038301M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


N04,  S04,  N08,  S08,  Nl2 


C038301 

CLASS 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0077 

SCHOOL  RECEIVE  CHAP  1 /TITLE  1 FUNDING? 
N04,  S04,  N08,  S08,  Nl2 


C038801 

CLASS 


001  C038801Y  (01 

002  C038801N  (02 

003  C038801M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0078 

DID  PRINCIPAL  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  N08,  S08,  N12 


C034101 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 
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001  C034101Y  (01  ) 0 

002  C034101M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0079 

DID  HEADMASTER/HEADMISTRESS  FILL  OUT  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  N12 

C034102  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034102Y  (01  ) 0 

002  C034102M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOOSO 

DID  HEAD  TEACHER  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  Nl2 

C034103  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034103Y  (01  ) 0 

002  C034103M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOOSl 

DID  VICE  PRINCIPAL  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  N12 

C034104  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  . NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034104Y  (01  ) 0 

002  C034104M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00S2 

DID  COUNSELOR  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  N12 

C034105  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034105Y  (01  ) 0 

002  C034105M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00S3 

DID  CURRICULUM  COORD  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  N12 

.C034106  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  . NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034106Y  (01  ) 0 

002  C034106M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00S4 

DID  TEACHER  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  N12 

C034107  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034107Y  (01  ) 0 

002  C034107M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL00S5 

DESCRIPTION:  DID  SECRETARY  FILL  OUT  THIS  QUESTIONNAIRE 

GRADES /ASSESSMENTS:  N04,  S04,  NOS,  SOS,  Nl2 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C 

TYPE  OF  CONTRAST:  C 

001  C03410SY  (01  ) 0 

002  C03410SM  (M  ) 1 


:03410S 

!LASS 


TUTALi  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


SCHL00S6 

DID  OTHER  PERSON  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  S04,  NOS,  SOS,  N12 

C034109  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C034109Y  (01  ) 0 

002  C034109M  (M  ) 1 

CONDITIONING  VARIABLE  ID:  TCHROOOl 

DESCRIPTION:  WHAT  IS  YOUR  GENDER? 

GRADES /ASSESSMENTS:  N04,  S04,  NOS,  SOS 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


O iO 

ERIC 
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CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T055901 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

001  T055901A  (01  ) 

00 

MALE 

002  T055901B  (02  ) 

10 

FEMALE 

003  T055901M  (M  ) 

01 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0002 

DESCRIPTION: 

WHICH  BEST  DESRIBES 

YOU? 

GRADES/ASSESSMENTS : 

N04,  S04, 

NOS , 

SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

7 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

6 

001  T056001A  (01  ) 

000000 

WHITE 

002  T056001B  (02  ) 

100000 

BLACK 

003  T056001C  (03  ) 

010000 

HISPANIC 

004  T056001D  (04  ) 

001000 

ASIAN/PACIFIC  AMERIC 

005  T056001E  (05  ) 

000100 

AMER  IND/ALASKA  NATV 

006  T056001F  (06  ) 

000010 

OTHER 

007  T056001M  (M  ) 

000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0003 

DESCRIPTION: 

YEARS  TAUGHT 

GRADES/ASSESSMENTS : 

N04,  S04, 

NOS , 

SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T040301 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

6 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

5 

001  T040301A  (01  ) 

00000 

2 YEARS  OR  LESS 

002  T040301B  (02  ) 

10000 

3-5  YEARS 

003  T040301C  (03  ) 

01000 

6-10  YEARS 

004  T040301D  (04  ) 

00100 

11-24  YEARS 

005  T040301E  (05  ) 

00010 

25  YEARS  OR  MORE 

006  T040301M  (M  ) 

00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0004 

DESCRIPTION: 

HOW  MANY 

YRS  TOTAL 

YOU  TAUGHT 

MATH? 

GRADES/ASSESSMENTS : 

N04,  S04 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056101 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

6 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

5 

001  T056101A  (01  ) 

1 00000 

2 YEARS  OR  LESS 

002  T056101B  (02  ) 

1 10000 

3-5  YEARS 

003  T056101C  (03  ] 

1 01000 

6-10  YEARS 

004  T056101D  (04  1 

1 00100 

11-24  YEARS 

005  T056101E  (05  ] 

1 00010 

25  YEARS  OR  MORE 

006  T056101M  (M  ] 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0005 

DESCRIPTION: 

HOW  MANY 

YRS  TOTAL 

YOU  TAUGHT 

SCIENCE? 

GRADES/ASSESSMENTS : 

N04.  S04 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056102 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

6 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

5 

001  T056102A  (01 

) 00000 

2 YEARS  OR  LESS 

002  T056102B  (02 

) 10000 

3-5  YEARS 

003  T056102C  (03 

) 01000 

6-10  YEARS 

004  T056102D  (04 

) 00100 

11-24  YEARS 

005  T056102E  (05 

) 00010 

25  YEARS  OR  MORE 

006  T056102M  (M 

) 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0006 

DESCRIPTION: 

TYPE  TCHNG  CERT  IN 

THIS  ST 

IN 

MAIN  FIELD? 

GRADES /ASSESSMENTS : 

N04,  S04, 

NOS, 

SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

7 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

6 

001  T056201A  (01 

) 000000 

ADVANCED  PROFESSIONL 

002  T056201B  (02 

) 100000 

REGULAR/STANDARD  ST 

003  T056201C  (03 

) 010000 

PROBATIONARY  STATE 

004  T056201D  (04 

) 001000 

TEMPORARY / PROVI S lONL 

005  T056201E  (05 

) 000100 

OTHER  THAN  STATE  CRT 

006  T056201F  (06 

) 000010 

NOT  HAVE  CERT  MAIN 

007  T056201M  (M 

) 000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0007 

DESCRIPTION: 

CERTIFICATION, 

ELEMENTARY 

OR 

MIDDLE/JUNIOR  HS  ED? 

GRADES /ASSESSMENTS : 

N04,  S04, 

, NOS, 

SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T040501 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

4 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

3 

ERIC 
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BEST  COPY  AVAILABtfi 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


001  T040501Y  (01 

002  T040501N  (02 

003  T040501C  (03 

004  T040501M  (M 


) 000 
) 100 
) 010 
) 001 


YES 

NO 

NOT  OFFERED 
MISSING 


IN  STATE 


CONDITIONING  VARIABLE  ID:  TCHR0008 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040506Y  (01 

002  T040506N  (02 

003  T040506C  (03 

004  T040506M  (M 


DO  YOU  HAVE  CERTIFICATION  IN  ELEMENTARY  MATH? 
N04,  S04,  NOS,  SOS 


T040506 

CLASS 

) 000  • 

) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0009 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  T040504 

TYPE  OF  CONTRAST:  CLASS 


DO  YOU  HAVE  CERTIFICATION  IN  JR  HIGH/ SEC  MATH? 
N04,  S04,  NOS,  SOS 


001  T040504Y  (01 

002  T040504N  (02 

003  T040504C  (03 

004  T040504M  (M 


) 000 
) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID:  TCHROOlO 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040507Y  (01 

002  T040507N  (02 

003  T040507C  (03 

004  T040507M  (M 


CERT I F I CAT I ON , ELEMENTARY  SCI ENCE ? 
N04,  NOS,  SOS 


T040507 

CLASS 

) 000 
) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

NO 

NOT  OFFERED 
MISSING 


IN  STATE 


CONDITIONING  VARIABLE  ID:  TCHROOll 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T04050SY  (01 

002  T04050SN  (02 

003  T04050SC  (03 

004  T04050SM  (M 


CERTIFICATION,  MIDDLE/ JUNIOR  SCIENCE 
N04,  NOS,  SOS 


T04050S 

CLASS 

) 000 
) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040505Y  (01  ) 

002  T040505N  (02  ) 

003  T040505C  (03 

004  T040505M  (M 


TCHR0012 
CERTIFICATION,  OTHER 
N04,  S04,  NOS,  SOS 

T040505 

CLASS 

) 000 
) 100 
) 010 
) 001 


CONDITIONING  VARIABLE  ID:  TCHR0013 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


HIGHEST  ACADEMIC  DEGREE  YOU  HOLD? 
N04,  S04,  NOS,  SOS 

T056301 
CLASS 


CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056301A  (01  ] 

1 0000000 

HIGH  SCHOOL  DIPLOMA 

002  T056301B  (02  ] 

1 1000000 

ASSOCIATES/VOCATIONL 

003  T056301C  (03  ] 

1 0100000 

BACHELORS  DEGREE 

004  T056301D  (04  ] 

1 0010000 

MASTER'S  DEGREE 

005  T056301E  (05  ] 

1 0001000 

EDUCATION  SPECIALIST 

006  T056301F  (06  ] 

1 0000100 

DOCTORATE 

007  T056301G  (07  ] 

1 0000010 

PROFESSIONAL  DEGREE 

OOS  T056301M  (M  ] 

1 0000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0014 

DESCRIPTION: 

EDUCATION  UNDERGRAD  MAJOR 

GRADES /ASSESSMENTS : 

N04,  S04,  NOS,  SOS 

T040701 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


O 512 

ERIC 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST: 

001  T040701Y  (01  ) 

002  T040701M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 

001  T040706Y  (01  ) 

002  T040706M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040707Y  (01  ) 

002  T040707M  (M  ) 


CLASS 

0 

1 

TCHR0015 

ELMENT  ED  UNDERGRAD  MAJOR 
N04,  S04,  NOB,  SOB 

T040706 

CLASS 

0 

1 

TCHR0016 

SEC  ED  UNDERGRAD  MAJOR 
N04,  S04,  NOB,  SOB 

T040707 

CLASS 

0 

1 


NUMBER  OF  INDEPENDENT  CONTRASTS; 
YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0017 

WAS  YOUR  UNDERGRADUATE  MAJOR  MATH? 

N04,  S04,  NOB,  SOB 

T040703  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS ; 


001  T040703Y  (01  ) 0 

002  T040703M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHROOIB 

WAS  YOUR  UNDERGRADUATE  MAJOR  I^TH  ED? 

N04,  S04,  NOB,  SOB 

T040704  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T040704Y  (01  ) 0 

002  T040704M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0019 

SCIENCE  ED  UNDERGRAD  MAJOR 
N04,  NOB,  SOB 

T040710 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040710Y  (01  ) 0 

002  T040710M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0020 

DESCRIPTION;  LIFE  SCIENCE  UNDERGRAD  MAJOR? 

GRADES /ASSESSMENTS:  N04,  NOB,  SOB 

CONDITIONING  VAR  LABEL: 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


NAEP  ID:  T040711 

TYPE  OF  CONTRAST:  CLASS 


001  T040711Y  (01  ) 0 

002  T040711M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0021 

DESCRIPTION:  PHYSICAL  SCIENCE  UNDERGRAD  MAJOR? 

GRADES /ASSESSMENTS:  N04,  NOB,  SOB 

CONDITIONING  VAR  LABEL: 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


NAEP  ID:  T040712 

TYPE  OF  CONTRAST:  CLASS 


001  T040712Y  (01  ) 0 

002  T040712M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0022 

DESCRIPTION:  EARTH  SCIENCE  UNDERGRAD  MAJOR? 

GRADES /ASSESSMENTS;  N04,  NOB,  SOB 

CONDITIONING  VAR  LABEL: 

NAEP  ID;  T040713  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST:  CLASS  NUMBER  OF  INDEPENDEINT  CONTRASTS: 


001  T040713Y  (01  ) 0 YES 

002  T040713M  (M  ) 1 MISSING 

CONDITIONING  VARIABLE  ID:  TCHR0023 

DESCRIPTION;  SPECIAL  EDUCATION  UNDERGRAD  MAJOR 


1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 
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BEST  COPY  AVAILABLE 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


N04,  S04,  N08,  SOS 

T040708 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040708Y  (01  ) 0 

002  T040708M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0024 

BILINGUAL  ED/ESL  UNDERGRAD  MAJOR 
N04,  S04,  N08,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T040709 
CLASS  . 


001  T040709Y  (01  ) 0 

002  T040709M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0025 

OTHER  UNDERGRAD  MAJOR 
N04,  S04,  N08,  SOS 

T040705 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040705Y  (01  ) 0 

002  T040705M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040801Y  (01 

002  T040801M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040807Y  (01 

002  T040807M  (M 


TCHR0026 

EDUCATION  GRAD  MAJOR 
N04,  S04«  NOS,  SOS 

T040801 

CLASS 

) 0 
) 1 

TCHR0027 

ELEMENTARY  ED  GRAD  MAJOR 
N04,  S04,  NOS,  SOS 

T040807 

CLASS 

> 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0028 

SECONDARY  ED  GRAD  MAJOR 
N04,  S04,  NOS,  SOS 

T040808 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040808Y  (01  ) 0 

002  T040808M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0029 

DESCRIPTION:  WAS  YOUR  GRADUATE  MAJOR  MATHEMATICS? 

GRADES /ASSESSMENTS:  N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  T040803 

TYPE  OF  CONTRAST:  CLASS 

001  T040803Y  (01  ) 0 

002  T040803M  (M  ) 1 

TCHR0030 

WAS  YOUR  GRADUATE  MAJOR  MATH  ED? 

N04,  S04,  NOS,  SOS 

T040804  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T040804Y  (01  ) 0 

002  T040804M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


'040803 

!LASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0031 

SCIENCE  ED  GRAD  MAJOR? 
N04,  NOS,  SOS 

T040814 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040814Y  (01  ) 0 

002  T040814M  (M  ) 1 


YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


^ )14 

ERIC 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040815Y  (01  ) 

002  T040815M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040816Y  (01  ) 

002  T040816M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040817Y  (01  ) 

002  T040817M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040809Y  (01  ) 

002  T040809M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040810Y  (01  ) 

002  T040810M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0032 

.LIFE  SCIENCE  GRAD  MAJOR? 

N04,  N08,  S08 

T040815 

CLASS 

0 

1 

TCHR0033 

PHYSICAL  SCIENCE  GRAD  MAJOR? 
N04,  N08,  S08 

T040816 

CLASS 

0 

1 

TCHR0034 

EARTH  SCIENCE  GRAD  MAJOR? 
N04,  N08,  S08 

T040817 

CLASS 

0 

1 

TCHR0035 

SPECIAL  ED  GRAD  MAJOR 
N04,  S04,  N08,  S08 

T040809 

CLASS 

0 

1 

TCHR0036 

BILINGUAL  GRAD  MAJOR 
N04,  S04,  N08,  S08 

T040810 

CLASS 

0 

1 

TCHR0037 

ADMIN/SUPERVISION  GRAD  MAJOR 
N04,  S04,  N08,  S08 

T040811 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED ‘CONTRASTS : 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040811Y  (01  ) 0 

002  T040811M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0038 

CURRICULUM/ INSTRUCTION  GRAD  MAJOR? 

N04,  S04,  N08,  S08 

T040812  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040812Y  (01  ) 0 

002  T040812M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0039 

COUNSELING  GRAD  MAJOR? 
N04,  S04,  N08,  S08 

T040813 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040813Y  (01  ) 0 

002  T040813M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  I 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


: TCHR0040 

OTHER  GRAD  MAJOR 
N04,  S04,  N08,  S08 

T040805 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


56G 


BEST  COPY  AVAIL^(E 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


001  T040805Y  (01  ] 

002  T040805M  (M  ) 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0041 

NO  GRADUATE  STUDY 
N04,  S04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T040806  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T040806Y  (01  ) 

002  T040806M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0042 

UNDERGRAD/GRAD  MINOR  STUDY-EDUCATION 
N04,  S04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T056401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056401Y  (01  ) 

002  T056401M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0043 

UNDERGRAD/GRAD  MINOR  STUDY-ELEMENTARY  ED 
N04,  S04,  N08,  S08 

NAEP  ID : 

TYPE  OF  CONTRAST: 

T056402  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056402Y  (01  ) 

002  T056402M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0044 

UNDERGRAD/GRAD  MINOR  STUDY-SECONDARY  ED 
N04,  S04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T056403  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056403Y  (01  ) 

002  T056403M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0045 

UNDERGRAD /GRAD  MINOR  STUDY- MATHEMATICS 
N04,  S04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T056404  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056404Y  (01  ) 

002  T056404M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0046 

UNDERGRAD/GRAD  MINOR  STUDY-MATHEMATICS  ED 
N04,  S04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T056405  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056405Y  (01  ) 

002  T056405M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0047 

UNDERGRAD /GRAD  MINOR  STUDY-SCIENCE  ED 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T056413  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056413Y  (01  ) 

002  T056413M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

TCHR0048 

UNDERGRAD /GRAD  MINOR  STUDY-LIFE  SCIENCE 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

T056414  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  T056414Y  (01  ) 

002  T056414M  (M  ) 

0 ■ YES 

1 . MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

TCHR0049 

UNDERGRAD/GRAD  MINOR  STUDY- PHYSICAL  SCIENCE 
N04,  N08,  S08 
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CONDITIONING  VAR  LABEL; 

NAEP  ID:  T056415 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056415Y  (01  ) 0 

002  T056415M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID;  TCHR0050 

DESCRIPTION;  UNDERGRAD/GRAD  MINOR  STUDY-EARTH  SCIENCE 


" GRADES /ASSESSMENTS;  N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID;  T056416 

TYPE  OF  CONTRAST;  CLASS 

001  T056416Y  (01  ) 0 

002  T056416M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0051 

UNDERGRAD/GRAD  MINOR  STUDY-SPECIAL  ED 
N04,  S04.  NOS,  SOS 

T056406  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056406Y  (01  ) 0 

002  T056406M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


TCHR0052 

UNDERGRAD/GRAD  MINOR  STUDY-BILINGUAL  ED 
N04,  S04,  NOS,  SOS 

T056407  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056407Y  (01  ) 0 

002  T056407M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0053 

UNDERGRAD/GRAD  MINOR  STUDY-ADMIN  & SUPERVISION 
N04,  S04,  NOS,  SOS 

T05640S  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T05640SY  (01  ) 0 

002  T05640SM  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0054 

UNDERGRAD/ GRAD  MINOR  STUDY-CURRICULUM  & INSTRUC 
N04,  S04,  NOS,  SOS 

T056409  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056409Y  (01  ) 0 

002  T056409M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0055 

UNDERGRAD /GRAD  MINOR  STUDY-COUNSELING 
N04,  S04,  NOS,  SOS 

T056410  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056410Y  (01  ) 0 

002  T056410M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0056 

UNDERGRAD /GRAD  MINOR  STUDY-OTHER 
N04,  S04,  NOS,  SOS 

T056411  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056411Y  (01  ) 0 

002  T056411M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0057 

UNDERGRAD/GRAD  MINOR  STUDY-NONE 
N04,  S04,  NOS,  SOS 

T056412  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


001  T056412Y  (01  ) 0 

002  T056412M  (M  ) 1 


YES 

MISSING 
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CONDITIONING  VARIABLE  ID: 

TCHR0058 

DESCRIPTION: 

LAST  YR.  HOW  MUCH  TIME 

IN 

MATH/MATH  ED  SEM/WRKSHP? 

GRADES /ASSESSMENTS : 

N04,  S04,  N08,  SOS 

CONDITIONING  VAR  LABEL: 

4 

NAEP  ID: 

T056501 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056501A  (01 

) 00000 

NONE 

002  T056501B  (02 

> 10000 

LESS  THAN  6 HOURS 

003  T056501C  (03 

> 01000 

6-15  HOURS 

004  T056501D  (04 

> 00100 

16-35  HOURS 

005  T056501E  (05 

1 00010 

MORE  THAN  35  HOURS 

006  T056501M  (M 

) 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0059 

DESCRIPTION: 

LAST  YR,  HOW  MUCH  TIME 

IN 

SCI/SCI  ED  SEM/WRKSHPS? 

GRADES /ASSESSMENTS : 

N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T058101 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T058101A  (01  1 

1 00000 

NONE 

002  T058101B  (02  ! 

1 10000 

LESS  THAN  6 HOURS 

003  T058101C  (03  j 

» 01000 

6-15  HOURS 

004  T058101D  (04  ] 

1 00100 

16-35  HOURS 

005  T058101E  (05  ] 

1 00010 

MORE  THAN  35  HOURS 

006  T058101M  (M  ] 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0060 

DESCRIPTION: 

LAST  2 YRS,  HOW  MANY 

MATH/MATH  ED  UNIV  COURSES? 

GRADES /ASSESSMENTS : 

N04.  S04,  N08,  S08 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056601 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056601A  (01  ) 

' 00000 

NONE 

002  T056601B  (02  ) 

1 10000 

ONE 

003  T056601C  (03  ) 

' 01000 

TWO 

004  T056601D  (04  ) 

' 00100 

THREE 

005  T056601E  (05  ) 

1 00010 

FOUR  OR  MORE 

006  T056601M  (M  ) 

00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0061 

DESCRIPTION: 

LAST  2 YRS,  HOW  MANY 

SCI/SCI 

ED 

UNIV  COURSES?- 

GRADES/ASSESSMENTS : 

N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T058201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T058201A  (01  > 

00000 

NONE 

002  T058201B  (02  > 

10000 

ONE 

003  T058201C  (03  > 

01000 

TWO 

004  T058201D  (04  ) 

00100 

THREE 

005  T058201E  (05  > 

00010 

FOUR  OR  MORE 

006  T058201M  (M  ) 

00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0062 

DESCRIPTION: 

PAST  5 YRS,  TAKEN  COURSES/ IN 

PRO 

DEVP-TELECOMM  USE 

GRADES/ASSESSMENTS : 

N04,  S04,  N08,  S08 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056701 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056701Y  (01  ) 

0 

YES 

002  T056701M  (M  ) 

1 

MISSING 

6 

5 


6 

5 


6 

5 


6 

5 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0063 

PAST  5 YRS,  TAKEN  COURSES /IN  PRO  DEVP-TECH  USE 
N04.  S04,  NOS.  SOS 

T056702  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  ■ NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056702Y  (01  ) 0 

002  T056702M  (M  > 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 

TCHR0064 

DESCRIPTION: 

PAST  5 YRS.  TAKEN  COURSES/ IN 

PRO  DEVP-COOP 

INSTRCT 

GRADES /ASSESSMENTS : 

N04,  S04,  N08,  S08 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056703 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF 

INDEPENDENT  CONTRASTS: 

001  T056703Y  (01  ] 

1 0 

YES 

002  T056703M  (M  ) 

1 1 

MISSING 

2 

1 


CONDITIONING  VARIABLE  ID:  TCHR0065 
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DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


PAST  5 YRS,  COURSES/IN  PRO  DEVLP-INTERDISP  INSTRCT 
N04,  S04,  NOS,  SOS 


T056704 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056704Y  (01 

002  T056704M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0066 

PAST  5 YRS,  COURSES/IN  PRO  DEVLP-PORTFOLIO  ASSMNT 
N04,  S04,  NOS,  SOS 


T056705 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056705Y  (01 

002  T056705M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0067 

PAST  5 YRS,  COURSES/ IN  PRO  DEVLP-PERF  BASED  ASSMNT 
N04,  S04,  NOS,  SOS 


T056706 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056706Y  (01 

002  T056706M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR006S 

PAST  5 YRS,  COURSES/PRO  DEVLP-TEACH  HIGHORDER  THKG 
N04,  S04,  NOS,  SOS 


T056707 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056707Y  (01 

002  T056707M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0069 

PAST  5 YRS,  COURSES/PRO  DEVLP-TEACH  DIFF  CULT  BKGD 
N04,  S04,  NOS,  SOS 


T05670S 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T05670SY  (01 

002  T05670SM  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0070 

PAST  5 YRS,  COURSES/PRO  DEVLP-TEACH  LEP  STUDENTS 
N04,  S04,  NOS,  SOS 


T056709 

CLASS 


TOTAL, NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056709Y  (01 

002  T056709M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0071 

PAST  5 YRS,  COURSES/PRO  DEVLP-TEACH  SPEC  NEED  STDS 
N04,  S04,  NOS,  SOS 


T056710 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056710Y  (01 

002  T056710M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0072 

PAST  5 YRS,  COURSES/PRO  DEVLP-CLASSRM  MNGMT/ORG 
N04,  S04,  NOS,  SOS 


T056711 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056711Y  (01 

002  T056711M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0073 

PAST  5 YRS,  COURSES/PRO  DEVLP-OTHER  PROF  ISSUES 
N04,  S04,  NOS,  SOS 


T056712 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


ERIC 


001  T056712Y  (01 


) 0 


BEST  COPY  AVAILABLE 
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002  T056712M  (M  ) 1 


MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0074 

PAST  5 YRS,  COURSES/ PRO  DEVLP-NONE  OF  ABOVE 
N04,  S04,  NOS,  SOS 

T056713  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056713Y  (01  ) 0 

002  T056713M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0075 

AVAILABILITY  OF  RESOURCES 
N04,  S04,  NOS,  SOS 

T041201 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T041201A  (01  ) 0000 

002  T041201B  (02  ) 1000 

003  T041201C  (03  ) 0100 

004  T041201D  (04  ) 0010 

005  T041201M  (M  ) 0001 


GET  ALL  RESOURCES 
GET  MOST  RESOURCES 
GET  SOME  RESOURCES 
DONT  GET  RESOURCES 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0076 

ARE  CURRICULUM  SPECIALISTS  AVAILABLE  FOR  MATH? 

N04,  S04 

T041302  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T041302Y  (01  ■ ) 00 

002  T041302N  (02  ) 10 

003  T041302M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0077 

DESCRIPTION:  SCIENCE  CURRICULUM  SPECIALIST 

GRADES/ASSESSMENTS:  N04,  S04 

CONDITIONING  VAR  LABEL: 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


NAEP  ID:  T041303 

TYPE  OF  CONTRAST:  CLASS 


001  T041303Y  (01  ) 00 

002  T041303N  (02  ) 10 

003  T041303M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0078 

DESCRIPTION:  HOW  MANY  SCHOOL  HOURS  ARE  PREP  TIME  PER  WEEK? 


GRADES/ASSESSMENTS : 

N04,  S04 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

T056801 

TYPE  OF  CONTRAST: 

CLASS 

001  T056S01A 

(01 

) 000000 

002  T056S01B 

(02 

) 100000 

003  T056S01C 

(03 

) 010000 

004  T056S01D 

(04 

) 001000 

005  T056S01E 

(05 

) 000100 

006  T056801F 

(06 

) 000010 

007  T056801M 

(M 

) 000001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

LESS  THAN  ONE 

1-2 

3-4 

5 

MORE  THAN  5 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0079 

METHODS  OF  TEACHING  ELEM  MATH-  1+COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

T056901  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056901Y  (01  ) 0 

002  T056901M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRI PTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


TCHROOSO 

METHODS  OF  TEACHING  ELEM  MATH-PART  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

T0569A1  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569A1Y  (01  ) 0 

002  T0569A1M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TCHROOSl 

METHODS  OF  TEACHING  ELEM  MATH-SEMINAR 
N04,  S04,  NOS,  SOS 

T0569B1  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


2 

1 


3 

2 


7 

6 


2 

1 


2 

1 


2 
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TYPE  OF  CONTRAST; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS ; ’ 1 

001  T0569B1Y  <01  ) 

002  T0569B1M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

TCHR0082, 

METHODS  OF  TEACHING  ELEM  MATH-LITTLE  NO  EXPOSURE 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST; 

T0569C1  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS ; 1 

001  T0569C1Y  (01  ) 

002  T0569C1M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

TCHR00S3 

NUMBER  SYSTEMS  & NUMERATION- 1+  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST; 

T056902  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  T056902Y  (01  ) 

002  T056902M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

TCHR00S4 

NUMBER  SYSTEMS  & NUMERATION- PART  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST; 

T0569A2  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS ; 1 

001  T0569A2Y  (01  ) 

002  T0569A2M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

TCHR00S5 

NUMBER  SYSTEMS  & NUMERATION-SEMINAR 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST; 

T0569B2  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

class  number  of  INDEPENDENT  CONTRASTS;  1 

001  T0569B2Y  (01  ) 

002  T0569B2M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

TCHR00S6 

NUMBER  SYSTEMS  & NUMERATION- LITTLE/ NO  EXPOSURE 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST; 

T0569C2  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

class  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  T0569C2Y  (01  ) 

002  T0569C2M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 

TCHR00S7 

MEASUREMENT  IN  MATH-  1+COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST: 

T056903  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

class  number  of  independent  CONTRASTS;  1 

001  T056903Y  (01  ) 

002  T056903M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

TCHROOSS 

MEASUREMENT  IN  MATH-  PART  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

NAEP  ID; 

TYPE  OF  CONTRAST: 

T0569A3  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

class  number  of  INDEPENDENT  CONTRASTS;  1 

001  T0569A3Y  (01  ) 

002  T0569A3M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 

TCHR00S9 

MEASUREMENT  IN  MATH-  -SEMINAR 
N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

T0569B3  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

class  number  of  INDEPENDENT  CONTRASTS;  1 

001  T0569B3Y  (01  1 

002  T0569B3M  (M  1 

1 0 yes 

1 1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

measurement  in  math-  -little  no  exposure  best  copy  available 
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GRADES/ASSESSMENTS:  N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 


NAEP  ID: 

T0569C3 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T0569C3Y  (01 

) 0 

YES 

002  T0569C3M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID; 

TCHR0091 

DESCRIPTION; 

GEOMETRY-  1+COLLEGE  COURSE 

GRADES /ASSESSMENTS ; 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 
NAEP  ID; 

T056904 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T056904Y  (01 

) 0 

YES 

002  T056904M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID; 

TCHR0092 

DESCRIPTION; 

GEOMETRY-PART  COLLEGE  COURSE 

GRADES /ASSESSMENTS ; 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 

NAEP  ID; 

T0569A4 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPEND^T  CONTRASTS: 

001  T0569A4Y  (01 

) 0 

YES 

002  T0569A4M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID; 

TCHR0093 

DESCRIPTION; 

GEOMETRY- SEMINAR 

GRADES /ASSESSMENTS ; 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 
NAEP  ID; 

T0569B4 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T0569B4Y  (01  ] 

1 • 0 

YES 

002  T0569B4M  (M  ] 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID; 

TCHR0094 

DESCRIPTION: 

GEOMETRY-LITTLE  NO  EXPOSURE 

GRADES/ASSESSMENTS : 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T0569C4 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS ; 

001  T0569C4Y  (01  ) 

0 

YES 

002  T0569C4M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0095 

DESCRIPTION: 

PROBABILITY/STATISTICS-  1+COLLEGE  COURSE 

GRADES /ASSESSMENTS : 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T056905 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T056905Y  (01  ) 

0 

YES 

002  T056905M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0096 

DESCRIPTION; 

PROBABILITY/STATISTICS-PART  COLLEGE  COURSE 

GRADES /ASSESSMENTS : 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 
NAEP  ID; 

T0569A5 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T0569A5Y  (01  ) 

0 

YES 

002  T0569A5M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID; 

TCHR0097 

DESCRIPTION; 

PROBABILITY/STATISTICS-SEMINAR 

GRADES /ASSESSMENTS ; 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID; 

T0569B5 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS ; 

001  T0569B5Y  (01  ) 

0 

YES 

002  T0569B5M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID; 

TCHR009S 

DESCRIPTION: 

PROBABILITY/STATISTICS-LITTLE  NO 

EXPOSURE 

GRADES/ASSESSMENTS ; 

N04,  S04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 

NAEP  ID; 

T0569C5 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T0569C5Y  (01  ) 

0 

YES 

002  T0569C5M  (M  ) 

1 

MISSING 

2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 
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CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0099 

COLLEGE  ALGEBRA-  1+COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 


T056906 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056906Y  (01  ) 0 

002  T056906M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES / AS SES SMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROlOO 

COLLEGE  ALGEBRA- PART  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

T0569A6  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CUiSS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569A6Y  (01  ) 0 

002  T0569A6M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROlOl 

COLLEGE  ALGEBRA -SEMINAR 
N04,  S04,  NOS,  SOS 

T0569B6 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569B6Y  (01  ) 0 

002  T0569B6M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0102 

COLLEGE  ALGEBRA- LITTLE  NO  EXPOSURE 
N04,  S04,  NOS,  SOS 


T0569C6 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  T0569C6Y  (01  ) 0 

002  T0569C6M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0103 

CALCULUS-  1+COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

T056907 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056907Y  (01  ) 0 

002  T056907M  (M  > 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /AS SES SMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0104 

CALCULUS -PART  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 

T0569A7 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569A7Y  (01  ) 0 

002  T0569A7M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T0569B7Y  (01  ) 

002  T0569B7M  (M  . ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0105 

CALCULUS - SEMINAR 
N04,  S04,  NOS,  SOS 

T0569B7 

CLASS 

0 

1 

TCHR0106 

CALCULUS-LITTLE  NO  EXPOSURE 
N04,  S04,  NOS,  SOS 

T0569C7 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569C7Y  (01  ) 0 

002  T0569C7M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0107 

ABSTRACT/LINEAR  ALGEBRA-  1+COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 


T05690S 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 
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001  T056908Y  (01  ) 0 YES 

002  T056908M  (M  ) 1 MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0108 

ABSTRACT/LINEAR  ALGEBRA-PART  COLLEGE  COURSE 
N04,  S04,  NOS,  SOS 


T0569A8 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569A8Y  (01  ) 0 

002  T0569A8M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0109 

ABSTRACT/LINEAR  ALGEBRA-SEMINAR 
N04,  S04,  NOS,  308 

T0569B8 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  T0569B8Y  (01  ) 0 

002  T0569B8M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROllO 

ABSTRACT/LINEAR  ALGEBRA-LITTLE  NO  EXPOSURE 
N04,  S04,  NOS,  SOS 


T0569C8 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T0569C8Y  (01  ) 0 

002  T0569C8M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROlll 

EVER  STUDIED  ESTIMATION? 
N04,  S04,  NOS,  SOS 

T057001 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057001Y  (01  ) 00 

002  T057001N  (02  ) 10 

003  T057001M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0112 

EVER  STUDIED  PROBLEM  SOLVING  IN  MATH? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T057002 

CLASS 


001  T057002Y  (01  ) 00 

002  T057002N  (02  ) 10 

003  T057002M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIpNING  VARIABLE  ID:  TCHR0113 

DESCRIPTION:  EVER  STUDIED  USE  OF  MANIPULATIVES’> 

GRADES /ASSESSMENTS:  N04,  S04,  NOS.  SOS 


CONDITIONING  VAR  LABEL: 


NAEP  ID:  T057003 

TYPE  OF  CONTRAST:  CLASS 

001  T057003Y  (01  ) 00 

002  T057003N  (02  ) 10 

003  T057003M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0114 

DESCRIPTION:  EVER  STUDIED  USE  OF  CALCULATORS  IN  MATH  INSTRC? 

GRADES /ASSESSMENTS:  N04,  S04,  NOS.  SOS 


CONDITIONING  VAR  LABEL: 


NAEP  ID:  T057004 

TYPE  OF  CONTRAST;  CLASS 

001  T057004Y  (01  ) 00 

002  T057004N  (02  ) 10 

003  T057004M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0115 

EVER  STUDIED  UNDERSTANDING  STUDS  MATH  THINKING? 

N04,  S04,  NOS,  SOS 

T057005  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057005Y  (01  ) 00 

002  T057005N  (02  ) 10 


YES 

NO 
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003  T057005M  (M  ) 01 


MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0116 

EVER  STUDIED  GENDER  ISSUES  IN  TEACHING  MATH? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T057006 

CLASS 


001  T057006Y  (01  ) 00 

002  T057006N  (02  ) 10 

003  T057006M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0117 

EVER  STUDIED  TEACHING  STUDS  OF  DIFF  CULTURES? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T057007 

CLASS 


001  T057007Y  (01  ) 00 

002  T057007N  (02  ) 10 

003  T057007M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROllS 

KNOWLEDGE  OF  NCTM  CURR  & EVAL  STANDARDS  FOR  MATH? 
N04,  S04,  NOS,  SOS 


T057101 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057101A  (01  ) 0000 

002  T057101B  (02  ) 1000 

003  T057101C  (03  ) 0100 

004  T057101D  (04  ) 0010 

005  T057101M  (M  ) 0001 


VERY  KNOWLEDGEABLE 
KNOWLEDGEABLE 
SOMEWHAT  KNOWLEDGEAB 
LITTLE/NO  KNOWLEDGE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0119 

PRO  ACTVTS- STRATEGIES  LOCAL  WORKSHOPS 
N04,  S04,  NOS,  SOS 


T057201 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057201Y  (01  ) 0 

002  T057201M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0120 

PRO  ACTVTS -STRATEGIES  REGIONAL  NCTM  MEETING 
N04,  S04,  NOS,  SOS 


T057211 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057211Y  (01  ) 0 

002  T057211M  (M  ) 1 


YES 

MISSING 


3 

2 


5 

4 


2 

1 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


TCHR0121 

PRO  ACTVTS- STRATEGIES  NATIONAL  NCTM  MEETING 
N04,  S04,  NOS,  SOS 


T057221 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057221Y  (01 

002  T057221M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


) 0 • 

) 1 

TCHR0122 

PRO  ACTVTS-STRATEGIES  OTHER 
N04,  S04,  NOS,  SOS 

T057231 

CLASS 


YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057231Y  (01  ) 0 

002  T057231M  (M  ) 1 


YES 

MISSING 


2 

1 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0123 

PRO  ACTVTS-STRATEGIES  NO 
N04,  S04,  NOS,  SOS 

T057241 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 


001  T057241Y  (01  ) 0 

002  T057241M  (M  ) 1 


YES 

MISSING 


’BT  COPY  AVAILABLE 


57G 


525 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUBOOOl 

IMPORTANCE  W/  STUDS-APPLYING  MATH  IDEAS? 
N04,  S04,  NOS,  SOS 


T057301 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057301A  (01  ) 0000 

002  T057301B  (02  ) 1000 

003  T057301C  (03  ) OlOO 

004  T057301D  (04  ) 0010 

005  T057301M  (M  ) 0001 


VERY  IMPORTANT 
SOMEWHAT  IMPORTANT 
NOT  VERY  IMPORTANT 
NOT  IMPORTANT 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0002 

IMPORTANCE  W/  STUDS -PROB  S0LVING=G0AL  & CONCEPT? 
N04,  S04,  NOS,  SOS 


T057302 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057302A  (01  ) 0000 

002  T057302B  (02  ) 1000 

003  T057302C  (03  ) OlOO 

004  T057302D  (04  ) 0010 

005  T057302M  (M  ) 0001 


VERY  IMPORTANT 
SOMEWHAT  IMPORTANT 
NOT  VERY  IMPORTANT 
NOT  IMPORTANT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0003 

IMPORTANCE  W/  STUDS-?  TECHS  PROMOTE  STUD  TALK’ 
N04,  S04,  NOS,  SOS 


T057303 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057303A  (01  ) 0000 

002  T057303B  (02  ) 1000 

003  T057303C  (03  ) 0100 

004  T057303D  (04  ) 0010 

005  T057303M  (M  ) 0001 


VERY  IMPORTANT 
SOMEWHAT  IMPORTANT 
NOT  VERY  IMPORTANT 
NOT  IMPORTANT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0004 

IMPORTANCE  W/  STUDS -USE  RESULTS  TO  INFORM  DECISION 
N04,  S04,  NOS,  SOS 


T057304 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057304A  (01  ) 0000 

002  T057304B  (02  ) 1000 

003  T057304C  (03  ) 0100 

004  T057304D  (04  ) 0010 

005  T057304M  (M  ) 0001 


VERY  important 
SOMEWHAT  IMPORTANT 
NOT  VERY  IMPORTANT 
NOT  IMPORTANT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0005 

TO  ACCESS  PROGRESS  HOW  OFTEN  USE  MULT  CHOICE  TESTS 
N04,  S04,  NOS,  SOS 


T057401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057401A  (01  ) 0000 

002  T057401B  (02  ) 1000 

003  T057401C  (03  ) 0100 

004  T057401D  (04  ) 0010 

005  T057401M  (M  ) 0001 


ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
ONCE  OR  TWICE  A YEAR 
NEVER /HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0006 

TO  ACCESS  PROGRESS  HOW  OFTEN  USE  PROBLEM  SETS 
N04,  S04,  NOS,  SOS 


T057402 

CLASS 


TOTAL  number  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057402A  (01  ) 0000 

002  T057402B  (02  ) 1000 

003  T057402C  (03  ) 0100 

004  T057402D  (04  ) 0010 

005  T057402M  (M  ) 0001 


ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
ONCE  OR  TWICE  A YEAR 
NEVER/ HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0007 

TO  ACCESS  PROGRESS  HOW  OFTEN  USE  SHORT  WRITTEN  RSP 
N04,  S04,  NOS,  SOS 


T057403 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T057403A  (01  ) 0000 

002  T057403B  (02  ) 1000 


ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 


Table  C-4  (continued) 
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003  T057403C  (03 

004  T057403D  (04 

005  T057403M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T057404A  (01 

002  T057404B  (02 

003  T057404C  (03 

004  T057404D  (04 

005  T057404M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 0100 
) 0010 
) 0001 


ONCE  OR  TWICE  A YEAR 
NEVER /HARDLY  EVER 
MISSING 


TSUB0008 

TO  ACCESS  PROGRESS  HOW  OFTEN  USE  INDV/GROUP  PRJCTS 
N04,  S04,  NOS,  SOS 


T057404 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
ONCE  OR  TWICE  A YEAR 
NEVER /HARDLY  EVER 
MISSING 


TSUB0009 

TO  ACCESS  PROGRESS  HOW  OFTEN  USE  PORTFOLIOS 
N04,  S04,  NOS,  SOS 


001  T057405A  (01 

002  T057405B  (02 

003  T057405C 

004  T057405D 

005  T057405M 


(03 

(04 

(M 


T057405 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
ONCE  OR  TWICE  A YEAR 
NEVER/HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TSUBOOlO  ■ 

BEST  DESCRIPTION  OF  COMPUTER  AVAILABLITY  IN  MATH 
N04,  S04,  NOS, 


SOS 


T057501 


TYPE  OF  CONTRAST: 

CLASS 

001  T057501A  (01  ) 

000000 

002  T057501B  (02  ) 

100000 

003  T057501C  (03  ) 

010000 

004  T057501D  (04  ) 

001000 

005  T057501E  (05  ) 

000100 

006  T057501F  (06  ) 

000010 

007  T057501M  (M  ) 

000001 

CONDITIONING  VARIABLE  ID: 

TSUBOOll 

DESCRIPTION: 

PRIMARY  USE  OF 

COMPUTERS 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS, 

SOS 

NAEP  ID: 

T057601 

TYPE  OF  CONTRAST: 

CLASS 

001  T057601A  (01  ) 

00000 

002  T057601B  (02  ) 

10000 

003  T057601C  (03  ) 

01000 

004  T057601D  (04  ) 

00100 

005  T057601E  (05  ) 

00010 

006  T057601M  (M  ) 

00001 

CONDITIONING  VARIABLE  ID: 

TSUB0012 

DESCRIPTION: 

ARE  STUDENTS  ASSIGNED  TO 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS, 

SOS 

NAEP  ID: 

T044002 

TYPE  OF  CONTRAST: 

CLASS 

001  T044002Y  (01  ] 

1 00 

002  T044002N  (02  ] 

1 10 

003  T044002M  (M  1 

I 01 

CONDITIONING  VARIABLE  ID: 

TSUB0013 

DESCRIPTION: 

IF  ASSIGNED  BY 

ABILITY, 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS, 

SOS 

NAEP  ID: 

T057701 

TYPE  OF  CONTRAST: 

CLASS 

001  T057701A  (01 

) 0000 

002  T057701B  (02 

) 1000 

003  T057701C  (03 

) 0100 

004  T057701D  (04 

) 0010 

005  T057701M  (M 

) 0001 

CONDITIONING  VARIABLE  ID: 

TSUB0014 

DESCRIPTION: 

IF  ASSIGNED  BY 

ABILITY, 

GRADES /ASSESSMENTS : 

N04,  S04,  NOS, 

SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE  AVAILABLE 
ONE  IN  CLASS 
2 OR  3 IN  CLASS 
4 OR  MORE  IN  CLASS 
DIFFICULT  TO  ACCESS 
EASY  TO  ACCESS 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

DRILL  AND  PRACTICE 
DEMO  OF  NEW  TOPICS 
PLAYING  MATH  GAMES 
STIMULATIONS/APPS 
DO  NOT  USE  COMPUTERS 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  BY  ABILITY 
BY  READING  ABILITY 
BY  MATH  ABILITY 
BY  GENERAL  ABILITY 
MISSING 


CONDITIONING  VAR  LABEL: 


BEST  copy  AVAILABLE 


ERIC 


578 


527 
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NAEP  ID: 

TYPE  OF  CONTRAST: 


T057801 

CLASS 


001  T057801A  (01 

) 00000 

002  T057801B  (02 

) 10000 

003  T057801C  (03 

) 01000 

004  T05780iD  (04 

) 00100 

005  T057801E  (05 

) 00010 

006  T057801M  (M 

) 00001 

CONDITIONING  VARIABLE  ID: 

TSUB0015 

DESCRIPTION: 

CREATE  GROUPS  IN  CLASS  FOR  MATH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08,  S08 

NAEP  ID: 

T044201 

TYPE  OF  CONTRAST: 

CLASS 

001  T044201Y  (01 

) 00 

002  T044201N  (02 

) 10 

003  T044201M  (M 

1 01 

CONDITIONING  VARIABLE  ID: 

TSUB0016 

DESCRIPTION: 

TIME/WEEK  ON  MATH  INSTRUTION? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08,  S08 

NAEP  ID: 

T044301 

TYPE  OF  CONTRAST: 

CLASS 

001  T044301A  (01  ) 

000 

002  T044301B  (02  ) 

100 

003  T044301C  (03  ) 

010 

004  T044301M  (M  ) 

001 

CONDITIONING  VARIABLE  ID: 

TSUB0017 

DESCRIPTION: 

HOW  MUCH  TIME  PER  WEEK  STUDS  DO  : 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04.  S04,  N08,  S08 

NAEP  ID: 

T057901 

TYPE  OF  CONTRAST: 

CLASS 

001  T057901A  (01  ) 

0000 

002  T057901B  (02  ) 

1000 

003  T057901C  (03  ) 

0100 

004  T057901D  (04  ) 

0010 

005  T057901M  (M  ) 

0001 

CONDITIONING  VARIABLE  ID: 

TSUB0018 

DESCRIPTION: 

AMOUNT  MATH  HOMEWORK  AS SIGN /DAY? 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08,  S08 

NAEP  ID: 

T044401 

TYPE  OF  CONTRAST: 

CLASS 

001  T044401A  (01  ) 

000000 

002  T044401B  (02  ) 

100000 

003  T044401C  (03  ) 

010000 

004  T044401D  (04  ) 

001000 

005  T044401E  (05  ) 

000100 

006  T044401F  (06  ) 

000010 

007  T044401M  (M  ) 

000001 

CONDITIONING  VARIABLE  ID: 

TSUB0019 

DESCRIPTION: 

HOW  OFTEN  DO  STUDENTS  DO  MATH  FRC 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  N08.  S08 

NAEP  ID: 

T044501 

TYPE  OF  CONTRAST: 

CLASS 

001  T044501A  (01  ) 

0000 

002  T044501B  (02  ) 

1000 

003  T044501C  (03  ) 

0100 

004  T044501D  (04  ) 

0010 

005  T044501M  (M  ) 

0001 

CONDITIONING  VARIABLE  ID: 

TSUB0020 

DESCRIPTION: 

HOW  OFTEN  DO  STUDENTS  DO  MATH  ON 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04.  S04.  N08,  S08 

NAEP  ID: 

T044502 

TYPE  OF  CONTRAST: 

CLASS 

001  T044502A  (01  ) 

0000 

002  T044502B  (02  ) 

1000 

003  T044502C  (03  ) 

0100 

004  T044502D  (04  ) 

0010 

005  T044502M  (M  ) 

0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  BY  ABILITY 
HIGH  ABILITY 
AVERAGE  ability 
LOW  ABILITY  . 

MIXED  ABILITY 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 1/2  HOURS  OR  LESS 
> 2 1/2  HRS-<  4 HRS 
4 HRS  OR  MORE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

NONE 

LESS  THAN  1/2  HOUR 
1/2-1  HOUR 
MORE  THAN  1 HOUR 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

15  MINUTES 
30  MINUTES 
45  MINUTES 
ONE  HOUR 

MORE  THAN  ONE  HOUR 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


o 


ERIC 


b7D 


Table  C-4  (continued) 
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O 

ERIC 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0021 

HOW  OFTEN  DO  STUDENTS  SOLVE  PROBS  W/  OTHER  STUDS? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T044512 

CLASS 


001  T044512A  (01  ) 0000 

002  T044512B  (02  ) 1000 

003  T044512C  (03  ) 0100 

004  T044512D  (04  ) 0010 

005  T044512M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


5 

4 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  T044513A  (01  ) 0000 

002  T044513B  (02  ) 1000 

003  T044513C  (03  ) 0100 

004  T044513D  (04  ) 0010 

005  T044513M  (M  ) 0001 


OBJECTS  LIKE  RULERS? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


TSUB0022 

HOW  OFTEN  DO  STUDENTS  WORK  W/ 
N04,  S04,  NOS,  SOS 

T044513 

CLASS 


5 

4 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


TSUB0023 

HOW  OFTEN  WORK  W/  COUNTING  BLOCKS . GEOMETRIC  SHAPES 
N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T044514 

CLASS 


001  T044514A  (01  ) 0000 

002  T044514B  (02  ) 1000 

003  T044514C  (03  ) 0100 

004  T044514D  (04  ) 0010 

005  T044514M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0024 

HOW  OFTEN  DO  STUDENTS  USE  A CALCULATOR? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T044505 

CLASS 


001  T044505A  (01  ) 0000 

002  T044505B  (02  > 1000 

003  T044505C  (03  ) 0100 

004  T044505D  (04  ) 0010 

005  T044505M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0025 ‘ 

HOW  OFTEN  DO  STUDENTS  TAKE  MATH  TESTS? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T044515 

CLASS 


001  T044515A  (01  ) 0000 

002  T044515B  (02  ) 1000 

003  T044515C  (03  ) 0100 

004  T044515D  (04  ) 0010 

005  T044515M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


5 

4 


5 

4 


CONDITIONING  VARIABLE  ID; 

TSUB0026 

DESCRIPTION: 

HOW  OFTEN 

DO  STUDENTS 

WRITE  ABOUT  PROBLEM-SOLVING? 

GRADES/ ASSESSMENTS : 

N04,  S04, 

NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T044507 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS : 

001  T044507A  (01  ) 

1 0000 

ALMOST  EVERY  DAY 

002  T044507B  (02  1 

1 1000 

ONCE  OR  TWICE  A WEEK 

003  T044507C  (03  ] 

1 0100 

ONCE  OR  TWICE  MONTH 

004  T044507D  (04  ] 

1 0010 

NEVER  OR  HARDLY  EVER 

005  T044507M  (M  ! 

» 0001 

MISSING 

CONDITIONING  VARIABLE  ID; 

TSUB0027 

DESCRIPTION: 

HOW  OFTEN 

DO  STUDENTS 

TALK  ABOUT  MATH  WORK? 

GRADES/ ASSESSMENTS : 

N04,  S04, 

NOS,  SOS 

CONDITIONING  VAR  LABEL; 

NAEP  ID: 

T044516 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T044516A  (01 

) 0000 

ALMOST  EVERY  DAY 

002  T044516B  (02 

) 1000 

ONCE  OR  TWICE  A WEEK 

BEST  COPY  AVAILABL§29 
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003  T044516C  (03  ) 0100 

004  T044516D  (04  ) 0010 

005  T044516M  (M  ) 0001 


ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0028 

HOW  OFTEN  DO  STUDENTS  WRITE  REPORTS/DO  PROJECTS? 
N04,  S04,  NOS,  SOS 


T044508 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T044508A  (01  ) 0000 

002  T044508B  (02  ) 1000 

003  T044508C  (03  ) 0100 

004  T044508D  (04  ) 0010 

005  T044508M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0029 

HOW  OFTEN  DO  STUDENTS  DISCUSS  MATH  W/OTHER  STDNTS? 
N04,  S04,  NOS,  SOS 


T044509 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T044509A  (01  ) 0000 

002  T044509B  (02  ) 1000 

003  T044509C  (03  ) 0100 

004  T044509D  (04  ) 0010 

005  T044509M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0030 

HOW  OFTEN  DO  STUDENTS  WORK  REAL-LIFE  MATH  PRBLMS? 
N04,  S04,  NOS,  SOS 


T044510 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T044510A  (01  ) 0000 

002  T044510B  (02  ) 1000 

003  T044510C  (03  ) 0100 

004  T044510D  (04  ) 0010 

005  T044510M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0031 

HOW  OFTEN  DO  STUDENTS  USE  A COMPUTER? 
N04,  S04,  NOS,  SOS 


T044506 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T044506A  (01  ) 0000 

002  T044506B  (02  ) 1000 

003  T044506C  (03  ) 0100 

004  T044506D  (04  ) 0010 

005  T044506M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST:. 


TSUB0032 

IN  MATH  CLASS  HOW  OFTEN  ADDRESS -NUMBERS  & OPS? 
N04,  S04,  NOS,  SOS 


T058001 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T058001A  (01  ) 0000 

002  T058001B  (02  ) 1000 

003  T058001C  (03  ) 0100 

004  T058001D  (04  ) 0010 

005  T058001M  (M  ) 0001 


A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0033 

IN  MATH  CLASS  HOW  OFTEN  ADDRESS-MEASUREMENT? 
N04,  S04,  NOS,  SOS 


T058002 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  T058002A  (01  ) 0000 

002  T058002B  (02  ) 1000 

003  T058002C  (03  ) 0100 

004  T058002D  (04  ) 0010 

005  T058002M  (M  ) 0001 


A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TSUB0034 

IN  MATH  CLASS  HOW  OFTEN  ADDRESS -GEOMETRY? 
N04,  S04,  NOS,  SOS 


T058003 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 


530 


O 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST:  CLASS 

001  T058003A  (01  ) 0000 

002  T058003B  (02  ) 1000 

003  T058003C  (03  ) 0100 

004  T058003D  (04  ) 0010 

005  T058003M  (M  ) 0001 


NUMBER  OF  INDEPENDENT  CONTRASTS: 

A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0035 

IN  MATH  CLASS  HOW  OFTEN  ADDRESS-DATA  ANALYSIS? 

N04,  S04,  N08,  S08 

T058004  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T058004A  (01  ) 0000 

002  T058004B  (02  ) 1000 

003  T058004C  (03  ) 0100 

004  T058004D  (04  ) 0010 

005  T058004M  (M  ) 0001 


A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0036 

IN  MATH  CLASS  HOW  OFTEN  ADDRESS -ALGEBRA  & FUNCT? 


N04,  S04,  N08,  S08 

T058005 

CLASS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T058005A  (01  ) 0000 

002  T058005B  (02  ) 1000 

003  T058005C  (03  ) 0100 

004  T058005D  (04  ) 0010 

005  T058005M  (M  ) 0001 


A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  T058006A  (01  ) 0000 

002  T058006B  (02  ) 1000 

003  T058006C  (03  ) 0100 

004  T058006D  (04  ) 0010 

005  T058006M  (M  ) 0001 


MATH  FACTS /CONCEPTS? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


ID:  TSUB0037 

IN  MATH  HOW  OFTEN  ADDRESS -LRN 
N04,  S04,  N08,  S08 

T058006 
CLASS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0038 

IN  MATH  HOW  OFTEN  ADDRESS-LRN  SKILLS/ PROCEDURES? 
N04,  S04,  N08,  S08 


T058007 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T058007A  (01  ) 0000 

002  T058007B  (02  ) 1000 

003  T058007C  (03  ) 0100 

004  T058007D  (04  ) 0010 

005  T058007M  (M  ) 0001 


A LOT 
SOME 

A LITTLE 

NONE 

missing 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  T058008A  (01  ) 0000 

002  T058008B  (02  ) 1000 

003  T058008C  (03  ) 0100 

004  T058008D  (04  ) 0010 

005  T058008M  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


ID:  -TSUB0039 

IN  MATH  HOW  OFTEN  ADDRESS -DEVELOP  REASONING  ABLTY? 
N04,  S04,  N08,  S08 

T058008 
CLASS 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001 

T058009A 

(01 

) 0000 

002 

T058009B 

(02 

) 1000 

003 

T058009C 

(03 

) 0100 

004 

T058009D 

(04 

) 0010 

005 

T058009M 

(M 

) 0001 

TO  COMMUNICATE  MATH? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

A LOT 
SOME 

A LITTLE 

NONE 

MISSING 


ID:  TSUB0040 

IN  MATH  HOW  OFTEN  ADDRESS-LRN 
N04,  S04,  N08,  S08 

T058009 
CLASS 


CONDITIONING  VARIABLE  ID:  TSUB0041 


4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


o 

ERIC 


582 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


DESCRIPTION:  DO  YOU  PERMIT  UNRESTRICTED  USE  OF  CALCULATORS'? 

GRADES /ASSESSMENTS:  N04,  S04,  NOS.  SOS 


CONDITIONING  VAR  LABEL: 


NAEP  ID:  T045401 

TYPE  OF  CONTRAST:  CLASS 

001  T045401Y  (01  ) 00 

002  T045401N  (02  ) 10 

003  T045401M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0042 

DO  YOU  PERMIT  USE  OF  CALCULATORS  ON  TESTS? 

N04,  S04,  NOS,  SOS 

T044S01  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  T044S01Y  (01  ) 00 

002  T044S01N  (02  ) 10 

003  T044S01M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0043 

STUDENTS  HAVE  ACCESS  TO  SCHL-OWNED  CALCULATORS? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T045001 

CLASS 


001  T045001Y  (01  ) 00 

002  T045001N  (02  ) 10 

003  T045001M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0044 

DO  YOU  PROVIDE  INSTRUCTION  IN  USE  OF  CALCULATORS? 

N04,  S04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T044901 

CLASS 


001  T044901Y  (01  ) 00 

002  T044901N  (02  ) 10 

003  T044901M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0045 

HOW  PREPARED  TO  TEACH  MATH  CONCEPTS? 
N04,  S04,  NOS,  SOS 


T045304 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T045304A  (01  ) 0000 

002  T045304B  (02  ) 1000 

003  T045304C  (03  ) 0100 

004  T045304D  (04  ) 0010 

005  T045304M  (M  ) 0001 


VERY  WELL  PREPARED 
MODERATELY  PREPARED 
NOT  VERY  PREPARED 
NOT  AT  ALL  PREPARED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0046 

HOW  PREPARED  TO  TEACH  MATH  PROCEDURES? 

N04,  S04,  NOS,  SOS 

T045305  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T045305A  (01  ) 0000 

002  T045305B  (02  ) 1000 

003  T045305C  (03  ) 0100 

004  T045305D  (04  ) 0010 

005  T045305M  (M  ) 0001 


VERY  WELL  PREPARED 
MODERATELY  PREPARED 
NOT  VERY  PREPARED 
NOT  AT  ALL  PREPARED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0047 

HOW  PREPARED  TO  TEACH  USE  OF  COMPUTERS? 


N04,  S04,  NOS,  SOS 

T045302 

CLASS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T045302A  (01  ) 0000 

002  T045302B  (02  ) 1000 

003  T045302C  (03  ) 0100 

004  T045302D  (04  ) 0010 

005  T045302M  (M  ) 0001 


VERY  WELL  PREPARED 
MODERATELY  PREPARED 
NOT  VERY  PREPARED 
NOT  AT  ALL  PREPARED 
MISSING 


CONDITIONING  VARIABLE  I 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


: TSUB0048 

HOW  PREPARED  TO  TEACH 
N04,  S04,  NOS,  SOS 

T045303 


USE  OF  CALCULATORS? 

TOTAL  NUMBER 


OF  SPECIFIED  CONTRASTS: 


3 

2 


3 

2 


3 

2 


3 

2 


5 

4 


5 

4 


5 

4 


5 


^532 

ERIC 


583 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST: 


CLASS 


001  T045303A  (01  ) 0000 

002  T045303B  (02  ) 1000 

003  T045303C  (03  ) 0100 

004  T045303D  (04  ) 0010 

005  T045303M  (M  ) 0001 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0049 

NUMBER  OF  STUDENTS  IN  CLASS? 
N04,  S04,  NOB,  SOB 

T044000 

CLASS 


001  T044000A  (01  ) 00000 

002  T044000B  (02  ) 10000 

003  T044000C  (03  ) 01000 

004  T044000D  (04  ) 00100 

005  T044000E  (05  ) 00010 

006  T044000M  (M  ) 00001 


NUMBER  OF  INDEPENDENT  CONTRASTS: 

VERY  WELL  PREPARED 
MODERATELY  PREPARED 
NOT  VERY  PREPARED 
NOT  AT  ALL  PREPARED 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

1-20  STUDENTS 
21-25  STUDENTS 
26-30  STUDENTS 
31-35  STUDENTS 
36  OR  MORE  STUDENTS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00B7 

BEST  DESCRIBES  HOW  BTH  GRADES  ARE  ORGANIZED? 

NOB,  SOB 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


C034201 

CLASS 


001  C034201A  (01  ) 000 

002  C034201B  (02  ) 100 

003  C034201C  (03  ) 010 

004  C034201M  (M  ) 001 


SELF-CONTAINED 
SEMI -DEPARTMENTALIZE 
DEPARTMENTALIZED 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHLOOBB 

DESCRIPTION:  ARE  BTH-GRADERS  ASSIGNED  TO  MATH  BY  ABILITY? 


GRADES /ASSESSMENTS : NOB,  SOB 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C034402 

TYPE  OF  CONTRAST:  CLASS 

001  C034402Y  (01  > 00 

002  C034402N  (02  ) 10 

003  C034402M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00B9 

ARE  BTH-GRADERS  ASSIGNED  TO  SCIENCE  BY  ABILITY? 
NOB,  SOB 


C034403 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034403Y  (01  ) 00 

002  C034403N  (02  ) 10 

003  C034403M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0090 

ARE  BTH-GRADERS  ASSIGNED  TO  ENGLISH  BY  ABILITY? 

NOB,  SOB 

C034401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CijssS  ’ NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034401Y  (01  ) 00 

002  C034401N  (02  ) 10 

003  C034401M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0091 

DESCRIPTION:  ARE  BTH-GRADERS  ASSIGNED  TO  ARTS  BY  ABILITY? 


GRADES /ASSESSMENTS:  NOB,  SOB 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C034406 

TYPE  OF  CONTRAST:  CLASS 

001  C034406Y  (01  ) 00 

002  C034406N  (02  > 10 

003  C034406M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0092 

HOW  OFTEN  BTH  GRDS  RECEIVE  COMP  SCI  INSTRUCTION? 
NOB,  SOB 


C034510 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034510A  (01  ) 00000 

002  C034510B  (02  ) 10000 


EVERY  DAY 

3 OR  4 TIMES  A WEEK 


4 


6 

5 


4 

3 


3 

2 


3 

2 


3 

2 


3 

2 


6 

5 


BEST  COPY  AVAILABLE 


533 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


003  C034510C  (03 

004  C034510D  (04 

005  C034510E  (05 

006  C034510M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C034511A  (01 


) 01000 
) 00100 
) 00010 
) 00001 


ONCE  OR  TWICE/WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  not  taught 
MISSING 


SCHL0093 

HOW  OFTEN  8TH  GRDS  RECEIVE  MATH  INSTRUCTION? 
NOS,  SOS 


C034511 

CLASS 


) 00000 


002  C034511B  (02 

) 10000 

003  C034511C  (03 

) 01000 

004  C034511D  (04 

) 00100 

005  C034511E  (05 

) 00010 

006  C034511M  (M 

) 00001 

CONDITIONING  VARIABLE  ID: 

SCHL0094 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

C034512 

TYPE  OF  CONTRAST: 

CLASS 

001  C034512A  (01 

) 00000 

002  C034512B  (02 

) 10000 

003  C034512C  (03 

) 01000 

004  C034512D  (04 

) 00100 

005  C034512E  (05 

) 00010 

006  C034512M  (M 

) 00001 

CONDITIONING  VARIABLE  ID: 

SCHL0095 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

C034513 

TYPE  OF  CONTRAST: 

CLASS 

001  C034513A  (01  ] 

1 00000 

002  C034513B  (02  ) 

1 10000 

003  C034513C  (03  ) 

' 01000 

004  C034513D  (04  ) 

1 00100 

005  C034513E  (05  ) 

00010 

006  C034513M  (M  ) 

00001 

CONDITIONING  VARIABLE  ID: 

SCHL0096 

DESCRIPTION: 

HOW  OFTEN  , 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

C034514 

TYPE  OF  CONTRAST: 

CLASS 

001  C034514A  (01  ) 

00000 

002  C034514B  (02  ) 

10000 

003  C034514C  (03  ) 

01000 

004  C034514D  (04  ) 

00100 

005  C034514E  (05  ) 

00010 

006  C034514M  (M  ) 

00001 

CONDITIONING  VARIABLE  ID: 

SCHL0097 

DESCRIPTION: 

HAS  ENGLISF 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

C031611 

TYPE  OF  CONTRAST: 

CLASS 

001  C031611Y  (01  ) 

00 

002  C031611N  (02  ) 

10 

003  C031611M  (M  ■ ) 

01 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE/WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  not  taught 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE/WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  not  taught 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE/WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE/WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


A PRIORITY? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

yes 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL009S 

SCHOOL  OFFER  STH  GR  STUDS  ALGEBRA  FOR  HS  CREDIT? 
NOS,  SOS 


C034601 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034601Y  (01  ) 00 

002  C034601N  (02  ) 10 

003  C034601M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


SCHL0099 

SCHOOL  W/  SPECIAL  FOCUS  ON  ENGLISH? 
NOS,  SOS,  N12 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


NAEP  ID:  C037203 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037203Y  (01  ) 0 

002  C037203M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOlOO 

SCHOOL  FOLLOW  DISTRICT/ STATE  ENGLISH  CURRICULUM? 

NOS,  SOS,  N12 

C037306  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037306Y  (01  ) 0 

002  C037306M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOlOl 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  MATH? 

NOS,  SOS 

C039401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  C039401Y  (01  ) 0 

002  C039401M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0102 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  SCIENCE? 

NOS,  SOS 

C039402  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  independent  CONTRASTS: 


001  C039402Y  (01  ) 0 

002  C039402M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0103 

DESCRIPTION:  SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  READING? 


GRADES /ASSESSMENTS:  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C039403 

TYPE  OF  CONTRAST:  CLASS 

001  C039403Y  (01  ) 0 

002  C039403M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


CONDITIONING  VARIABLE  ID:  SCHL0104 

DESCRIPTION:  SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  ARTS? 

GRADES/ ASSESSMENTS:  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

C039404 
CLASS 


NAEP  ID: 

TYPE  OF  CONTRAST: 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039404Y  (01  - ) 0 

002  C039404M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0105 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  OTHER? 

NOS,  SOS 

C039405  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

nTjvcc  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039405Y  (01  ) 0 

002  C039405M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0106 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  NONE  ABOVE? 

NOS,  SOS 

C039406  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039406Y  (01  ) 0 

002  C039406M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0107 

DESCRIPTION:  8TH  GRADERS  IN  EXTRACURR  ACTS  FOR  MATH? 


GRADES/ ASSESSMENTS:  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C039501 

TYPE  OF  CONTRAST:  CLASS 

001  C039501Y  (01  ) 0 

002  C039501M  (M  ) 1 


CONDITIONING  VARIABLE  ID:  SCHLOIOS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


BEST  copy  AVAILABLE 

535 


V- 


86 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


DESCRIPTION; 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


8TH  GRADERS  IN  EXTRACURR  ACTS  FOR  SCIENCE? 
NOS,  SOS 


C039502 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C039502Y  (01  ) 0 

002  C039502M  (M  ) 1 


YES 

MISSING 


ID:  SCHL0109 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  ENG/LANG  ARTS? 
NOS,  SOS 

CONDITIONING  VAR  LABEL; 

NAEP  ID:  C 

TYPE  OF  CONTRAST;  c 

001  C039503Y  (01  ) 0 

002  C039503M  (M  ) 1 


039503 

LASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


YES 

MISSING 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS; ' 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST: 


SCHLOllO 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  ARTS? 
NOS,  SOS 


C039504 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C039504Y  (01  ) 0 

002  C039504M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOlll 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  NONE  OF  ABOVE’ 
NOS,  SOS 


C039505 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039505Y  (01  ) 0 

002  C039505M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0112 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  MATH? 
NOS,  SOS 


C039601 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039601Y  (01  ) 0 

002  C039601M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0113 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  SCIENCE? 
NOS,  SOS 


C039602 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C039602Y  (01  ) 0 

002  C039602M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SCHL0114 

STH  GRADERS  IN  SUMMER  PROGRAMS 
NOS,  SOS 

C039603 

CLASS 


I ENG/ LANG  ARTS? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C039603Y  (01  ) 0 

002  C039603M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0115 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  ARTS? 
NOS,  SOS 


C039604 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS ; 


001  C039604Y  (01  ) 0 

002  C039604M  (M  ) 1 


yes' 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0116 

STH  GRADERS  IN  SUMMER  PROGRAMS 
NOS,  SOS 

C039605 

CLASS 


J NONE  OF  ABOVE? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039605Y  (01  ) 0 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


YES 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


002  C039605M  (M  ) 1 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS:  N08,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C041901 


MISSING 

SCHL0117 

WHAT  % OF  8TH  GRDS  HELD  BACK/REPEAT  8TH  GRADE? 
NOS, 


TYPE  OF  CONTRAST: 

CLASS 

001 

C041901A 

(01 

) 00000 

002 

C041901B 

(02 

) 10000 

003 

C041901C 

(03 

) 01000 

004 

C041901D 

(04 

) 00100 

005 

C041901E 

(05 

) 00010 

006 

C041901M 

(M 

) 00001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

0% 

1-2% 

3-5% 

6-10% 

MORE  THAN  10% 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T063001A  (01 

002  T063001B  (02 

003  T063001C  (03 

004  T063001D  (04 

005  T063001E  (05 

006  T063001M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0124 

YRS  TOTAL  TAUGHT  MATH 
NOS,  SOS 

T063001 

CLASS 

) 00000 
) 10000 
) 01000 
) 00100 
) 00010 
) 00001 


001  T058301Y  (01 

002  T058301N  (02 

003  T058301M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  C035002Y  (01 

002  C035002N  (02 

003  C035002M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C035003Y  (01 

002  C035003N  (02 

003  C035003M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C035006Y  (01 

002  C035006N  (02 

003  C035006M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C035007Y  (01 

002  C035007N  (02 

003  C035007M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 YEARS  OR  LESS 
3-5  YEARS 
6-10  YEARS 
11-24  YEARS 
25  YEARS  OR  MORE 
MISSING 


TCHR0125 

CURRICULUM  SPECIALIST  TO  HELP/ADVISE  IN  MATH? 
NOS,  SOS 


T058301 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0118 

ARE  12TH-GRADERS  ASSIGNED  TO  MATH  BY  ABILITY? 
N12 


C035002 

CLASS 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0119 

ARE  12TH-GRADERS  ASSIGNED  TO  SCIENCE  BY  ABILITY? 
N12 


C035003 

CLASS 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

■ MISSING 


SCHL0120 

ARE  12TH-GRADERS  ASSIGNED  TO  ENGLISH  BY  ABILITY? 
N12 


C035006 

CLASS 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0121 

ARE  12TH-GRADERS  ASSIGNED  TO  ARTS  BY  ABILITY? 
N12 


C035007 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0122 

FROM  9TH  ON  HOW  MANY  YRS  REQUIRED  FOR  MATH? 


BEST  COPY  AVAILABLE 


58S 


537 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040201 

TYPE  OF  CONTRAST: 

CLASS 

001  C040201A  (01 

) 0000000 

002  C040201B  (02 

1 1000000 

003  C040201C  (03 

) 0100000 

004  C040201D  (04 

) 0010000 

005  C040201E  (05  ] 

1 0001000 

006  C040201F  (06  ] 

I 0000100 

007  C040201G  (07  ] 

I 0000010 

008  C040201M  (M  ] 

1 0000001 

CONDITIONING  VARIABLE  ID: 

SCHL0123 

DESCRIPTION: 

FROM  9TH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040202 

TYPE  OF  CONTRAST: 

CLASS 

001  C040202A  (01  ) 

0000000 

002  C040202B  (02  ) 

1000000 

003  C040202C  (03  ) 

0100000 

004  C040202D  (04  ) 

0010000 

005  C040202E  (05  ) 

0001000 

006  C040202F  (06  ) 

0000100 

007  C040202G  (07  ) 

0000010 

008  C040202M  (M  ) 

0000001 

CONDITIONING  VARIABLE  ID: 

SCHL0124 

DESCRIPTION: 

PROM  9TH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040203 

TYPE  OF  CONTRAST: 

CLASS 

001  C040203A  (01  ) 

0000000 

.002  C040203B  (02  ) 

1000000 

003  C040203C  (03  ) 

0100000 

004  C040203D  (04  ) 

0010000 

005  C040203E  (05  ) 

0001000 

006  C040203F  (06  ) 

0000100 

007  C040203G  (07  ) 

0000010 

008  C040203M  (M  ) 

0000001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

ONE  HALF  YEAR 
ONE  YEAR 
TWO  YEARS 
THREE  YEARS 
FOUR  YEARS 

MORE  THAN  FOUR  YEARS 
MISSING 


REQUIRED  FOR  SCIENCE? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

NONE 

ONE  HALF  YEAR 
ONE  YEAR 
TWO  YEARS 
THREE  YEARS 
FOUR  YEARS 

MORE  THAN  FOUR  YEARS 
MISSING 


REQUIRED  FOR  ENG/LIT? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

NONE 

ONE  HALF  YEAR 
ONE  YEAR 
TWO  YEARS 
THREE  YEARS 
FOUR  YEARS 

MORE  THAN  FOUR  YEARS 
MISSING 


CONDITIONING  VARIABLE  ID: 

SCHL0125 

DESCRIPTION: 

FROM  9TH 

ON 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040204 

TYPE  OF  CONTRAST: 

CLASS 

001  C040204A  (01 

) 0000000 

002  C040204B  (02 

) 1000000 

003  C040204C  (03 

1 0100000 

004  C040204D  (04 

1 0010000 

005  C040204E  (05 

) 0001000 

006  C040204F  (06  ] 

1 0000100 

007  C040204G  (07  ] 

1 0000010 

008  C040204M  (M.  ] 

1 0000001 

CONDITIONING  VARIABLE  ID: 

SCHL0126 

DESCRIPTION: 

COURSES  1 

OR 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040301 

TYPE  OF  CONTRAST: 

CLASS 

001  C040301Y  (01  ) 

1 0 

002  C040301M  (M  ) 

^ 1 

CONDITIONING  VARIABLE  ID: 

SCHL0127 

DESCRIPTION: 

COURSES  1 

OR 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040302 

TYPE  OF  CONTRAST: 

CLASS 

001  C040302Y  (01  ) 

0 

002  C040302M  (M  ) 

1 

CONDITIONING  VARIABLE  ID; 

SCHL0128 

DESCRIPTION: 

COURSES  1 

OR 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040303 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

ONE  HALF  YEAR 
ONE  YEAR 
TWO  YEARS 
THREE  YEARS 
FOUR  YEARS 

MORE  THAN  FOUR  YEARS 
MISSING 


IN  ADVANCED  BIO? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


IN  ADVANCED  CHEM? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS : 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


IN  ADV  PHYSICS? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


8 

7 


8 

7 


8 

7 


8 

7 


2 

1 


2 

1 


2 


. ; \ ' 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040303Y  (01  ) 

002  C040303M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0129 

NO  ADVANCED  SCIENCE  COURSES  TAUGHT 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040304  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040304Y  (01  ) 

002  C040304M  (M  ) 

0 yes 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0130 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  COMP  SCI? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040305  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  C040305Y  (01  ) 

002  C040305M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0131 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  CALCULUS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040306  TOTAL  NUMBER  OP  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  C040306Y  (01  ) 

002  C040306M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0132 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  TRIGONOMETRY? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040307  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  C040307Y  (01  ) 

002  C040307M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0133 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  PRECALCULUS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040308  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: ■ 2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040308Y  (01  ] 

002  C040308M  (M  ] 

1 0 yes 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0134 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  PROB/STAT? 
N12 

NAEP  ID: 

TYPE  OP  CONTRAST: 

C040309  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040309Y  (01 

002  C040309M  (M 

) 0 yes 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0135 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  UNI/INTEG  MATH? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040310  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  independent  CONTRASTS:  1 

001  C040310Y  (01 

002  C040310M  (M 

) 0 yes 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0136 

NO  ADVANCED  MATH  COURSES  TAUGHT? 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C040311  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  op  independent  CONTRASTS:  1 

001  C040311Y  (01 

002  C040311M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  MATH?  BEST  COPY  AVAILABLE 

ERIC 

590 


539 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES /ASSESSMENTS;  N12 

CONDITIONING  VAR  LABEL; 

NAEP  ID;  C040401 

TYPE  OF  CONTRAST;  CLASS 

001  C040401Y  (01  ) 00 

002  C040401N  (02  ) 10 

003  C040401M  (M  ) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0138 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  SCIENCE'^ 
N12 


C040402 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C040402Y  (01  ) 00 

002  C040402N  (02  ) 10 

003  C040402M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0139 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  ENG/LANG  ARTS 
N12 


C040403 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C040403Y  (01  ) 00 

002  C040403N  (02  ) 10 

003  C040403M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0140 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  FINE/PERF  ART 
N12 


C040404 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C040404Y  (01  ) 00 

002  C040404N  (02  ) 10 

003  C040404M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0141 

SCHOOL  SPONSER  12 TH  GR  FIELD  TRIPS  FOR  MATH? 
N12 


C040501 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C040501Y  (01  ) 0 

002  C040501M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0142 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  FOR  SCIENCE? 
N12 


C040502 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C040502Y  (01  ) 0 

002  C040502M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0143 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  IN  ENG/ LANG  ART 
N12 


C040503 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C040503Y  (01  ) 0 

002  C040503M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0144 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  IN  ARTS’ 

N12 

CO 40504  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C040504Y  (01  ) 0 

002  C040504M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 


SCHL0145 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  IN  OTHER? 
N12 


3 

2 


3 

2 


3 

2 


3 

2 


2 

1 


2 

1 


2 

1 


2 

1 


C040505 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 


2 


Table  C-4  (continued) 

Conditioning  Variables  Specific  <to  the  1996  Mathematics  Assessment 


TYPE  OF  CONTRAST: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  . 1 

001  C040505Y  (01  ) 

002  C040505M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0146 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  IN  NONE  ABOVE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

CO 40 50 6 TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040506Y  (01  ) 

002  C040506M  (M  ‘ ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0147 

12TH  GRADERS  IN  EXTRACURR  ACTS  IN  MATH? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

CO 40 601  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040601Y  (01  ) 

002  C040601M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0148 

12TH  GRADERS  IN  EXTRACURR  ACTS  IN  SCIENCE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040602  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  C040602Y  (01  ) 

002  C040602M  (M  > 

0 ' YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0149 

12TH  GRADERS  IN  EXTRACURR  ACTS  IN  ENG /LANG  ARTS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040603  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  C040603Y  (01  > 

002  C040603M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0150 

12TH  GRADERS  IN  EXTRACURR  ACTS  IN  ARTS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040604  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040604Y  (01  ) 

002  C040604M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0151 

12 TH  GRADERS  IN  EXTRACURR  ACTS  IN  NONE  OF  ABOVE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040605  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  C040605Y  (01  ) 

002  C040605M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0152 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  MATH? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040701  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040701Y  (01  ) 

002  C040701M  (M  ) 

1 0 yes 

, 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0153 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  SCIENCE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040702  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040702Y  (01 

002  C040702M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

12TH  GRADERS  IN  SUMMER  PORGRAMS  IN  ENG /LANG  ARTS?  BEST  COPY  AVAILABLE 

er|c 

- bQ2 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C040703Y  (01 

002  C040703M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C040704Y  (01  ) 

002  C040704M  (M  ) 


N12 


C040703 

CLASS 


) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

missing 


SCHL0155 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  ARTS’ 
N12 


C040704 

CLASS 

0 

1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0156 

DESCRIPTION: 
grades /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C040705 

TYPE  OF  CONTRAST:  CLASS 


12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  NONE  OF  ABOVE? 
N12 


001  C040705Y  (01  ) 

002  C040705M  (M  ) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C040801 


SCHL0157 

# STUDS  ENROLLED  IN  AP  SCIENCE  COURSES? 
N12 


TYPE  OF  CONTRAST: 

CLASS 

001 

C040801A 

(01 

) 0000000 

002 

C040801B 

(02 

) 1000000 

003 

C040801C 

(03 

) 0100000 

004 

C040801D 

(04 

) 0010000 

005 

C040801E 

(05 

1 0001000 

006 

C040801F 

(06  ; 

I 0000100 

007 

C040801G 

(07  ] 

1 0000010 

008 

C040801M 

(M  ] 

1 0000001 

CONDITIONING 

VARIABLE  ID: 

SCHL0158 

DESCRIPTION: 

# STUDS  ] 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


C040802 


TYPE  OF  CONTRAST: 

CLASS 

001  C040802A  (01 

) 0000000 

002  C040802B  (02 

) 1000000 

003  C040802C  (03 

) 0100000 

004  C040802D  (04 

) 0010000 

005  C040802E  (05 

) 0001000 

006  C040802F  (06 

) 0000100 

007  C040802G  (07 

) 0000010 

008  C040802M  (M 

) 0000001 

CONDITIONING  VARIABLE  ID: 

SCHL0159 

DESCRIPTION: 

# STUDS  ] 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040803 

TYPE  OF  CONTRAST: 

CLASS 

001  C040803A  (01  ] 

1 0000000 

002  C040803B  (02  ] 

1 1000000 

003  C040803C  (03  ] 

1 0100000 

004  C040803D  (04  ) 

1 0010000 

005  C040803E  (05  ] 

1 0001000 

006  C040803F  (06  ) 

1 0000100 

007  C040803G  (07  ) 

' 0000010 

008  C040803M  (M  ) 

1 0000001 

CONDITIONING  VARIABLE  ID: 

SCHL0160 

DESCRIPTION: 

# STUDS  I 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

C040804 

TYPE  OF  CONTRAST: 

CLASS 

001  C040804A  (01  ) 

0000000 

002  C040804B  (02  ) 

1000000 

003  C040804C  (03  ) 

0100000 

004  C040804D  (04  ) 

0010000 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

NONE 

1-10 

11-25 

26-50 

51-75 

76-99 

100  OR  MORE 
MISSING  ■ 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

1-10 

11-25 

26-50 

51-75 

76-99 

100  OR  MORE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

1-10 

11-25 

26-50 

51-75 

76-99 

100  OR  MORE 

missing 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

1-10 

11-25 

26-50 


2 

1 


2 

1 


2 

1 


8 

7 


8 

7 


8 

7 


8 

7 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


005 

C040804E 

(05 

) 0001000 

006 

C040804F 

(06 

) 0000100 

007 

C040804G 

(07 

) 0000010 

008 

C040804M 

(M 

) 0000001 

51-75 

76-99 

100  OR  MORE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS  t 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0161 

ANY  12TH  GRDS  TAKING  COLLEGE  COURSES  IN  MATH? 

N12 

C040901  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


001  C040901Y  (01  ) 0 

002  C040901M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0162 

ANY  12TH  GRDS  TAKING  COLLEGE  COURSES  IN  SCIENCE? 

N12 

C040902  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C040902Y  (01  ) 0 

002  C040902M  (M  ) 1 


YES 

MISSING 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0163 

ANY  12TH  GRDS  TAKING  COLLEGE  COURSES  ENG/LANG  ARTS 
N12 

C040903  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C040903Y  (01  ) 0 

002  C040903M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES / ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0164 

ANY  12TH  GRDS  TAKING  COLLEGE  COURSES  IN  ARTS? 

N12 

C040904  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


2 

1 


001  C040904Y . (01  ) 0 

002  C040904M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0165 

ANY  12TH  GRDS  TAKING  COLLEGE  COURSES  IN  NONE  ABOVE 
N12 

C040905  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C040905Y  (01  ) 0 

002  C040905M  (M  ) 1 


YES 

MISSING 


CONDITIONING 

DESCRIPTION: 

VARIABLE  ID: 

SCHL0166 

WHAT  % 12TH  GRDS 

HELD  BACK  AND  REPEAT  12TH 

GRADE? 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

N12 

C041001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF 

INDEPENDENT  CONTRASTS: 

001  C041001A 

(01  ; 

1 00000 

0% 

002  C041001B 

(02  ; 

1 10000 

1-2% 

003  C041001C 

(03  ; 

1 01000 

3-5% 

004  C041001D 

(04  ; 

1 00100 

6-10% 

005  C041001E 

(05  ; 

1 00010 

MORE  THAN 

10% 

006  C041001M 

(M 

1 00001 

MISSING 

VARIABLE  ID:  SCHL0167 

LAST  YR  WHAT  % OF  12TH  GRDS  GRADUATED? 
N12 


CONDITIONING 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


NAEP 

TYPE 

' ID: 

; OF  CONTRAST: 

C041101 

CLASS 

001 

C041101A 

(01 

) 00000 

002 

C041101B 

(02 

) 10000 

003 

C041101C 

(03 

) 01000 

004 

C041101D 

(04 

) 00100 

005 

C041101E 

(05 

) 00010 

006 

C041101M 

(M 

) 00001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

99-100% 

95-98% 

90-94% 

75-89% 

LESS  THAN  75% 

MISSING 


6 

5 


CONDITIONING  VARIABLE  ID:  SCHL0168 

DESCRIPTION:  WHAT  % OF  GRADUATING  CLASS  NOW  IN  2-YR  COLLEGE? 

GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL: 


BEST  COPY  AVAILABLE 


-...094 


543 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


NAEP  ID: 

TYPE  OF  CONTRAST: 


C036001 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C036001A  (01 

) 000000 

0-10% 

002  C036001B  (02 

) 100000 

11-25% 

003  C036001C  (03 

) 010000 

26-50% 

004  C036001D  (04 

) 001000 

51-75% 

005  C036001E  (05 

) 000100 

76-90% 

006  C036001F  (06 

) 000010 

91-100% 

007  C036001M  (M 

) 000001 

MISSING 

conditioning  VARIABLE 

ID: 

SCHL0169 

DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

WHAT  % OF  graduating 
N12 

CLASS  NOW  IN  4-YR  COLLEGE? 

NAEP  ID: 

C036002 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036002A  (01 

) 000000 

0-10% 

002  C036002B  (02 

) 100000 

11-25% 

003  C036002C  (03 

) 010000 

26-50% 

004  C036002D  (04 

) 001000 

51-75% 

005  C036002E  (05 

) 000100 

76-90% 

006  C036002F  (06 

) 000010 

91-100% 

007  C036002M  (M 

) 000001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0170 

DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL 

WHAT  % OF  GRADUATING 
N12 

CLASS  NOW  IN  VO-TEC  SCHOOL? 

NAEP  ID: 

C036003 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036003A  (01 

1 000000 

0-10% 

002  C036003B  (02 

1 100000 

11-25% 

003  C036003C  (03 

1 

1 010000 

26-50% 

004  C036003D  (04 

1 

1 001000 

51-75% 

005  C036003E  (05 

1 

1 000100 

76-90% 

006  C036003F  (06 

1 

1 000010 

91-100% 

007  C036003M  (M 

1 

1 000001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0171 

DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL 

WHAT  % OF  GRAD  CLASS 
N12 

NOW  IN  EMPLOYER  TRAINING? 

NAEP  ID: 

C036004 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036004A  (01 

) 

1 000000 

0-10% 

002  C036004B  (02 

) 

100000 

11-25% 

003  C036004C  (03 

) 

1 010000 

26-50% 

004  C036004D  (04 

) 

001000 

51-75% 

005  C036004E  (05 

) 

000100 

76-90% 

006  C036004F  (06 

) 

000010 

91-100% 

007  C036004M  (M 

) 

000001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0172 

DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL 

% OF  GRADUATING  CLASS 
N12 

NOW  IN  MILITARY  SERVICE? 

NAEP  ID: 

C036005 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036005A  (01 

) 

000000 

0-10% 

002  C036005B  (02 

) 

100000 

11-25% 

003  C036005C  (03 

) 

010000 

26-50% 

004  C036005D  (04 

) 

001000 

51r75% 

005  C036005E  (05 

) 

000100 

76-90% 

006  C036005F  (06 

) 

000010 

91-100% 

007  C036005M  (M 

) 

000001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0173 

DESCRIPTION: 

PERCENT  OF  STUDENTS  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 

GRADES /ASSESSMENTS : 

N04,  S04,  N08,  SOS.  N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

NTLUNSC 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  NATLUNCM  (M 

) 

1 0 

MISSING 

002  NATLUNCH  (0 

) 

1 1 

PERCENT 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0174 

PERCENT  OF  STUDENTS  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N04,  S04,  NOS,  SOS.  N12 

NTLUNSC  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


O 44 

ERIC 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


001  NATLUNCL  ( 0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0175 

PERCENT  OF  STUDENTS  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N04,  S04,  NOS,  SOS,  N12 

REMRDSC  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMREADM  (M  ) 0 

002  REM  READ  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0176 

PERCENT  OF  STUDENTS  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N04,  S04,  NOS,  SOS,  N12 

REMRDSC  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMREADL  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0177 

PERCENT  OF  STUDENTS  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N04,  S04,  NOS,  SOS,  N12 

REMMHSC  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMMATHM  (M  ) 0 

002  REM  MATH  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL017S 

PERCENT  OF  STUDENTS  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N04,  S04,  NOS,  SOS,  N12 

REMMHSC  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMMATHL  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


2 

1 


1 

1 


2 

1 


1 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0179 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N04,  S04 

NTLUNGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 


001  NATLUN4M  (M  ) 0 

002  NATLUN4  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOISO 

PERCENT  OF  STUDENTS  IN  4TH.WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N04,  S04 

NTLUNGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 


001  NATLUN4L  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0181 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N04,  S04 

REMRDGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMREA4M  (M  ) 0 

002  REMREAD4  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0182 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N04,  S04 

REMRDGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMREA4L  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0183 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N04,  S04 

REMMHGR  TOT7VL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMMAT4M  (M  ) 0 


MISSING 


2 

1 


1 

1 


2 

1 
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Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


002  REMMATH4  (0 


) 1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS: 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0184 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N04«  S04 


REMMHGR 

LINEAR 


001  REMMAT4L  (0-100, M=:0  ) 0.0  + 1.0*X 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

TYPE  OF  CONTRAST:' 

001  NATLUN8M  (M  ) 

002  NATLUN8  (0  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSES SMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  NTLUNGR 

TYPE  OF  CONTRAST:  LINEAR 


SCHL0185 

PERCENT  OF. STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N08,  S08 


NTLUNGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MISSING 

PERCENT 


SCHL0186 

PERCENT  OF  STUDENTS  IN8TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N08,  S08 


001  NATLUN8L  ( 0-100, M=0  ) 0.0  + 1.0*X 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  REMREA8M  (M  ) 

002  REMREAD8  (0  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS:  • 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0187 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N08,  S08 


REMRDGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MISSING 

PERCENT 


SCHL0188 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N08,  S08 


REMRDGR 

LINEAR 


001  REMREA8L  ( 0-100, M=0  ) 0.0  + 1.0*X 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

LINEAR 


CONDITIONING  VARIABLE  ID: 

DESCRIPTION: 

GRADES/ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

TYPE  OF  CONTRAST: 

001  REMMAT8M  (M  ) 

002  REMMATH8  (0  ) 

CONDITIONING  VARIABLE  ID: 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  , REMMHGR 

TYPE  OF  CONTRAST:  LINEAR 

001  REMMAT8L  (0-100, M=0  ) 0.0  + 1.0*X 


SCHL0189 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N08,  S08 


REMMHGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


MISSING 

PERCENT 


SCHL0190 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N08,  S08 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  NATLU12M  (M  ) 

002  NATLUN12  (0  ) 


SCHL0190 

RCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N12 


NTLUNGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR-  LABEL: 

NAEP  ID:  NTLUNGR 

TYPE  OF  CONTRAST:  LINEAR 


SCHL0191 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  NATLU12L  (0-100, M=0  ) 0.0  + 1.0*X 


[6 

ERIC 


597 


Table  C-4  (continued) 

Conditioning  Variables  Specific  to  the  1996  Mathematics  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0192 

RCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMOTIAL  READING 
N12 

REMRDGR  TOTAL  NUMBER  OP  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OP  INDEPENDENT  CONTRASTS: 


001  REMRD12M  (M  ) 0 

002  REMRD12  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0193 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N12 

REMRDGR  TOTAL  NUMBER  OP  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OP  INDEPENDENT  CONTRASTS: 


001  REMRD12L  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0194 

RCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N12 

REMMHGR  TOTAL  NUMBER  OP  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  REMMH12M  (M  ) 0 

002  REMMH12  (@  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0195 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N12 

REMMHGR  TOTAL  NUMBER  OP  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OP  INDEPENDENT  CONTRASTS: 


001  REMMH12L  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


2 

1 


1 

1 


1 

1 


BEST  COPY  AVAIUSLi 


. ^ . 
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Table  C-5 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  I 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  OVERALL  (@ 

CONDITIONING  VARIABLE  I 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  MALE  (1 

002  FEMALE  (2 


BACKOOOl 

GRAND  MEAN 

N04,  NOS,  SOS,  N12 

OVERALL 

BKSER 

OTHER 

) 1 

BACK0002 

DERIVED  SEX 

N04,  NOS,  SOS,  N12 

GENDER 

DSEX 

CLASS 

) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

GRAND  MEAN 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

MALE 

FEMALE 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0003 

DERIVED  RACE/ETHNICITY 

N04,  NOS,  SOS,  N12 

RACE/ETH 

DRACE 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  4 

NUMBER  OF  INDEPENDENT  CONTRASTS:  3 


001 

WHI/AI/0 

(1,5,6 

) 000 

002 

BLACK 

(2 

) 100 

003 

HISPANIC 

(3 

) 010 

004 

ASIAN 

(4 

) 001 

RACE/ETHNICITY:  WHITE,  AMERICAN  INDIAN /ALAS KAN 

NATIVE,  OTHER,  MISSING,  UNCLASSIFIED 
RACE/ETHNICITY:  BLACK 

RACE/ETHNICITY:  HISPANIC 

RACE/ETHNICITY:  ASIAN 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0004 

IF  HISPANIC,  WHAT  IS  YOUR  HISPANIC  BACKGROUND? 

N04,  NOS,  SOS,  N12 
HISPANIC 

B003101  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001  NOT  HISP  (1  ) 0000 

002  MEXICAN  (2  ) 1000 

003  PUER  RIC  (3  ) 0100 

004  CUBN,OTH  (4,5.  ) 0010 

005  HISP-?  (M  ) 0001 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0005 

MSA/NON-MSA 

N04,  NOS,  SOS,  N12 

MSANAT 

TOLS 

CLASS 


HISPANIC 

HISPANIC 

HISPANIC 

HISPANIC 

HISPANIC 


NOT  HISPANIC 

MEXICAN,  MEXICAN  AMERICAN,  CHI CANO 
PUERTO  RICAN 
CUBAN,  OTHER 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 


001  MSA  (1,2, 3, 4, 7, 9)  0 MSA 

002  NON  MSA  (5,6, S ) 1 NON-MSA 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0006 

TYPE  OF  LOCALE  (5  CATEGORIES) 

N04,  NOS,  SOS,  N12 

TOL5 

TOL5 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001 

BIG  CTY5 

(1 

) 0000 

002 

MID  CTY5 

(2,M 

) 1000 

003 

FR/BTWN5 

(3 

) 0100 

004 

SML  TWN5 

(4 

) 0010 

005 

RURAL5 

(5 

) 0001 

TOL5:  LARGE  CITY 

TOL5:  MID-SIZE  CITY 

TOL5:  URBAN  FRINGE  OF  LARGE  CITY,  URBAN  FRINGE  OF 

MID-SIZE  CITY 

TOL5 : SMALL  TOWN 

TOL5:  RURAL  (MSA  AND  NON-MSA) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  BIG  CITY  (1 

002  URBAN  FR  (2 

003  MED  CITY  (3, 9 ,M 

004  SM  PLACE  (4 


BACK0007 

DESCRIPTION  OF  COMMUNITY 

N04,  NOS,  SOS,  N12 

DOC 

DOC  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  4 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  3 


000 

DOC: 

BIG  CITY 

100 

DOC: 

URBAN  FRINGE 

010 

DOC: 

MEDIUM  CITY 

001 

DOC: 

SMALL  PLACE 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


BACKOOOS 

PARENTS'  HIGHEST  LEVEL  OF  EDUCATION 

N04,  NOS,  SOS,  N12 

PARED 

pared  total  number  of  specified  CONTRASTS: 


5 


O 8 

ERIC 


^99 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 


CLASS 


NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  < HS  (1 

002  HS  GRAD  (2 

003  POST  HS  (3 

004  COL  GRAD  (4 

005  PARED-?  (5,M 


) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


CONDITIONING  VARIABLE  ID;  BACK0009 


DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  N EAST  (1 

002  S EAST  (2 

003  CENTRAL  (3 

004  WEST  (4,5 


REGION  OF  THE  COUNTRY 

N04,  NOS,  N12 

REGION 

REGION 

CLASS 


) 000 
) 100 
) 010 
) 001 


CONDITIONING  VARIABLE  ID;  BACKOOlO 


DESCRIPTION; 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  PUBLIC  (1 

002  PRIVATE  (2,4,5,M 

003  CATHOLIC  (3 


SCHOOL  TYPE  (PQ) 

N04,  NOS,  SOS,  N12 

SCHTYPE 

SCHTYPE 

CLASS 


) 00 
) 10 


) 01 


PARED;  LESS  THAN  HIGH  SCHOOL 
PARED;  high  SCHOOL  GRADUATE 
PARED;  POST  HIGH  SCHOOL 
PARED ; COLLEGE  GRADUATE 
PARED;  MISSING,  I DON'T  KNOW 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


REGION 

REGION 

REGION 

REGION 


NORTHEAST 

SOUTHEAST 

CENTRAL 

WEST,  TERRITORIES  (NONE) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


SCHOOL  TYPE;  PUBLIC 

SCHOOL  TYPE;  PRIVATE,  BUREAU  OF  INDIAN  AFFAIRS, 
DEPARTMENT  OF  DEFENSE,  MISSING 
SCHOOL  TYPE;  CATHOLIC 


CONDITIONING  VARIABLE  ID;  BACKOOll 


DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


001  lEP-YES 

002  lEP-NO 


(1 

(2 


INDIVIDUALIZED  EDUCATION  PROGRAM 

N04,  NOS,  SOS,  N12 

lEP 

lEP 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


lEP; 

lEP; 


YES 

NO 


CONDITIONING  VARIABLE  ID;  BACK0012 


DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


LIMITED  ENGLISH  PROFICIENCY 

N04,  NOS,  SOS,  N12 

LEP 

LEP 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  LEP-YES 

002  LEP-NO 


(1 

(2 


LEP; 

LEP; 


YES 

NO 


CONDITIONING  VARIABLE  ID;  BACK0013 


DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


001  CHAPl-Y 

002  CHAPl-N 


(1 

(2 


CHAPTER  1 (BOOK  COVER) 

N04,  NOS,  SOS,  N12 

CHAPTER 1 

CHAPl 

CLASS 

0 

1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


CHAPTER  1 ; YES 
CHAPTER  1 : NO 


CONDITIONING  VARIABLE  ID:  BACK0014 


o 

ERIC 


DESCRIPTION: 

GRADES /ASSESSMENTS : 


DO  YOU  RECEIVE  A FREE  OR  REDUCED-PRICE  LUNCH? 
N04,  NOS,  SOS,  N12 


CONDITIONING 

VAR  LABEL: 

LUNCH 

NAEP  ID; 

SLUNCH 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

5 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

4 

001  NOT  ELIG 

(1  1 

1 0000 

LUNCH  PROGRAM 

NOT  ELIGIBLE 

002  REDUCED 

(2  1 

1 1000 

LUNCH  PROGRAM 

REDUCED  PRICE 

003  FREE 

(3  ; 

1 0100 

LUNCH  PROGRAM 

FREE 

004  INFO  NA 

(4,m  ; 

1 0010 

LUNCH  PROGRAM 

INFO  NOT  AVAILABLE 

005  SCH  REF 

(5  ; 

1 0001 

LUNCH  PROGRAM 

SCHOOL  REFUAL 

CONDITIONING 

VARIABLE  ID; 

BACK0015 

DESCRIPTION; 

HOW  OFTEN 

DO  THE  PEOPLE 

IN  YOUR  HOME  SPEAK  A LANGUAGE  OTHER  THAN  ENGLISH? 

GRADES/ASSESSMENTS : 

N04,  NOS, 

SOS,  N12 

CONDITIONING 

VAR  LABEL: 

HOMELANG 

NAEP  ID: 

B003201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

4 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

3 

001  HL-NEVER 

(1  1 

) 000 

HOMELANG : NEVER 

002  HL-SOME 

(2  1 

1 100 

HOMELANG : SOMET IMES 

003  HL-ALWAY 

(3  1 

) 010 

HOMELANG : ALWAYS 

004  HL-? 

(M  1 

1 001 

HOMELANG:  MISSING 

EST  COPY  AVAILABLE 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  ID:  BACK0016 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


HOW  MUCH  TV/VIDEO  DO  YOU  USUALLY  WATCH  EACH  DAY?  (LINEAR) 

N04,  NOS,  SOS,  n12 

TVWATCHL 


B009001 

LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

TVLIN-0 

(1 

) 0 

002 

TVLIN-1 

(2 

) 1 

003 

TVLIN-2 

(3 

) 2 

004 

TVLIN-3 

(4,M 

) 3 

005 

TVLIN-4 

(5 

) 4 

006 

'TVLIN-5 

(6 

) 5 

007 

TVLIN-6 

(7 

) 6 

TV  watching 
TV  WATCHING 
TV  WATCHING 
TV  WATCHING 
TV  WATCHING 
TV  watching 
TV  WATCHING 


(LINEAR) 

(LINEAR) 

(LINEAR) 

(LINEAR) 

(LINEAR) 

(LINEAR) 

(LINEAR) 


(0  TO  6+  HOURS  PER  DAY) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  TV-QUAD  (1-7,M=4 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  HW-MISS  (M 

002  HW-NO  (1 

003  HW-YES  (2-5 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0017 

HOW  MUCH  TV/ VIDEO  DO  YOU  USUALLY  WATCH  EACH  DAY?  (QUADRATIC) 

N04,  N08,  S08,  N12 
TVWATCHQ 

B009001  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

QUADRATIC  NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 1.0  + -2.0*X  + 1.0*X* 


TV  WATCHING  (QUADRATIC) 


BACK0018 

HOMEWORK  ASSIGNED?: 

N04,  N08,  S08,  N12 

HWASSIGN 

B006601 

CLASS 


) 00 
) 10 
) 01 


BASED  ON  TIME  SPENT  ON  HOMEWORK  EACH  DAY. 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

HOMEWORK  assigned?:  MISSING 

HOMEWORK  assigned?:  NO 

HOMEWORK  assigned?:  yES 


BACK0019 

HOW  MUCH  TIME  DO  YOU  USUALLY  SPEND  ON  HOMEWORK  EACH  DAY?  (LINEAR) 
N04,  N08,  S08,  N12 
HOMEWRKL 
B006601 
LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  HWLIN-0  (1,2,M 


) 0 


002  HWLIN-1 

003  HWLIN-2 

004  HWLIN-3 


(3 

(4 

(5 


HOMEWORK  (LINEAR) : 
MISSING 

HOMEWORK  (LINEAR) : 
HOMEWORK  (LINEAR) : 
HOMEWORK  (LINEAR) : 


DON'T  HAVE  ANY,  DON'T  DO  ANY, 

1/2  HOUR  OR  LESS 
1 HOUR 

MORE  THAN  1 HOUR 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NAEP  ID: 

TYPE  OF  CONTRAST: 

001  HWQUAD-0  (1,2,M  ) 0 


) 1 
) 2 
) 3 

BACK0020 

HOW  MUCH  TIME  DO  YOU  USUALLY  SPEND  ON  HOMEWORK  EACH  DAY  (QUADRATIC) 

N04,  N08,  S08,  N12 

HOMEWRKQ 

B006601  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

SCALE  NUMBER  OF  INDEPENDENT  CONTRASTS: 


002  HWQUAD-1 

003  HWQUAD-2 

004  HWQUAD-3 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


1 

4 

9 

BACK0021 

HOURS  EXTRA  READING/WK, 
N04,  N08,  S08 

B009101 


HOMEWORK  (QUADRATIC) : 
ANY,  missing 
HOMEWORK  (QUADRATIC) : 
HOMEWORK  (QUADRATIC) : 
HOMEWORK  (QUADRATIC) : 


NOT  CONNECTED  W/  SCHOOL? 


DON'T  HAVE  ANY,  DON'T  DO 

1/2  HOUR  OR  LESS 
1 HOUR 

MORE  THAN  1 HOUR 


TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS:  7 

001  NONE 

(01  ] 

1 0000000 

NONE 

002  1-2  HRS 

(02  ] 

1 1000000 

1-2  HOURS 

003  3-4  HRS 

(03  ] 

1 0100000 

3-4  HOURS 

004  5-6  HRS 

(04  ] 

1 0010000 

5-6  HOURS 

005  7-8  HRS 

(05  ] 

1 0001000 

7-8  HOURS 

006  9-10  HRS 

(06  ] 

1 0000100 

9-10  HOURS 

007  > 10  HRS 

(07  ] 

1 0000010 

MORE  THAN  10  HOURS 

008  B009101M 

(M  ] 

1 0000001 

MISSING 

CONDITIONING 

VARIABLE  ID: 

BACK0022 

DESCRIPTION: 

NUMBER  OF 

ITEMS  IN  THE  HOME 

(NEWSPAPER,  >25  BOOKS,  ENCYCLOPEDIA,  MAGAZINES) 

(DERIVED) 

GRADES /ASSESSMENTS : 

N04,  N08, 

S08,  N12 

CONDITIONING 

VAR  LABEL: 

HOMEITMS 

NAEP  ID: 

HOMEEN3 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS:  2 

001  HITEM<=2 

(1,M  ) 

1 00 

ITEMS  IN  HOME:  ZERO  TO  TWO  ITEMS,  MISSING 

002  HITEM=s3 

(2  ) 

1 10 

ITEMS  IN  HOME:  THREE  ITEMS 

O 
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003  HITEM=4  (3 


) 01 


ITEMS  IN  HOME:  FOUR  ITEMS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0023 

HOW  MANY  DAYS  OF  SCHOOL  MISSED  LAST  MONTH? 
N04,  N08.  SOS.  N12 
SCH  MISS 
S004001 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  MISS->2 

002  MISS-2< 


(3.4.5.M  ) 0 

(1.2  ) 1 

BACK0024 

HOW  MANY  GRADES  IN  THIS  STATE 
NOS.  SOS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades/ ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  B007602 

TYPE  OF  CONTRAST:  CLASS 

001  < ONE  GR  (01  ) 0000 

002  1-2  GRDS  (02  ) 1000 

003  3-5  GRDS  (03  ) 0100 

004  > 5 GRDS  (04  ) 0010 

005  MISSING  (M  ) 0001 


DAYS  OF  SCHOOL  MISSED: 

DAYS.  MISSING 

DAYS  OF  SCHOOL  MISSED: 


3-4.  5-10.  10  OR  MORE 
0-1.  2 DAYS 


(4) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

LESS  THAN  ON  GRADE 
1-2  GRADES 
3-5  GRADES 
5 OR  MORE  GRADESEVER 
MISSING 


CONDITIONING  VARIABLE  ID:  BACK0025 


DESCRIPTION: 
grades/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SINCE  1ST  GR. 
NOS.  SOS 


BOIOIOI 

CLASS 


NOT  PROMOTION,  HOW  OFTEN  DIFF  SCHLS? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

BOIOIOIA 

(01  ) 

1 00000000 

NONE 

002 

BOIOIOIB 

(02  ) 

1 10000000 

1 

003 

BOIOIOIC 

(03  ) 

1 01000000 

2 

004 

BOIOIOID 

(04  ] 

1 00100000 

3 

005 

BOIOIOIE 

(05  ) 

1 00010000 

4 

006 

BOIOIOIF 

(06  ) 

1 00001000 

5 

007 

BOIOIOIG 

(07  ) 

1 00000100 

6 

OOS 

BOIOIOIH 

(OS  ) 

1 00000010 

7 OR  MOREMORE 

009 

BO 10 10 IM 

(M  ] 

1 00000001 

MISSING 

CONDITIONING 

VARIABLE  ID: 

BACK0026 

DESCRIPTION: 

HOW  LONG  LIVED 

IN  THE  UNITED  STATES? 

grades/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SOS.  N12 


001  AALL  LIF 

002  USA  >5 

003  USA  3-5 

004  USA  <3 

005  USA-? 


(1 

(2 

(3 

(4 

(M 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  STGRD<1  (l.M 

002  STGRDl-2  (2 

003  STGRD3>  (3 


N04.  NOS. 
YRSINUSA 
B008001 
CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 

BACK0027 

HOW  MANY  GRADES 

N04 

STGRADE4 

B007601 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YEARS 


ALL  MY  LIFE 

LIVED  IN  US  MORE  THAN  5 
LIVED  IN  US  3-5  YEARS 
LIVED  IN  US  LESS  THAN  3 YEARS 
LIVED  IN  US  MISSING 


IN  THIS  STATE?  (4TH  GRADE) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 00 
) 10 
) 01 


GRADES  IN  STATE 
GRADES  IN  STATE 
GRADES  IN  STATE 


LESS  THAN  1 GRADE.  MISSING 
1-2  GRADES 
3 OR  MORE  GRADES 


CONDITIONING  VARIABLE  ID:  BACK0029 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TIMES  CHANGED  SCHOOL  SINCE  FIRST  GRADE 
N12 


001 

002 

003 

004 

005 

006 
007 
OOS 
009 


B007302A  (01 
B007302B  (02 
B007302C  (03 
B007302D  (04 
B007302E  (05 
B007302F  (06 
B007302G  (07 
B007302H  (OS 
B007302M  (M 


B007302 

CLASS 

) 00000000 
) 10000000 
) 01000000 
) 00100000 
) 00010000 
) 00001000 
) 00000100 
) 00000010 
) 00000001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE  . 

1 

2 

3 

4 

5 

6 

7 OR  MORE 
missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 


BACK0030 

HOW  MANY  GRADES  IN  THIS  STATE  (12TH  GRADE) 
N12 


BEST  COPY  AVAILABLE 
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CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  STGRD<1  (1,M 

002  STGRDl-2  (2 

003  STGRD3-5  (3 

004  STGRD6>  (4,5 


• STGRAD12 
B008301 
CLASS 

) 000 
) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


GRADES  IN  STATE 
GRADES  IN  STATE 
GRADES  IN  STATE 
GRADES  IN  STATE 


LESS  THAN  1 GRADE,  MISSING 

1-2  GRADES 

3-5  GRADES 

MORE  THAN  5 GRADES 


CONDITIONING  VARIABLE  ID:  SCHLOOOl 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SCHNRM-?  (M 

002  SCHNRM-Y  (@ 


SCHOOL  LEVEL  AVERAGE  SCIENCE  NORMIT  (MISSING  VS  NON-MISSING) 

N04,  NOS,  SOS,  N12 
SCH  NORM 

SCHNORM  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


SCHOOL  LEVEL  AVERAGE  SCIENCE  NORMIT  MISSING  • 
SCHOOL  LEVEL  AVERAGE  SCIENCE  NORMIT  NOT -MIS SING 


CONDITIONING  VARIABLE  ID:  SCHL0002 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SNRM-LIN  (# 

002  SNRM-LIN  (M 


SCHOOL  LEVEL  AVERAGE  SCIENCE  NORMIT 

N04,  NOS,  SOS,  N12 

SNRM-LIN 

SCHNORM  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

SCALE  NUMBER  OF  INDEPENDENT  CONTRASTS: 


) (FS.4) 

) 0 


SCHOOL  LEVEL  AVERAGE  SCIENCE  NORMIT  MEAN 
SCHOOL  LEVEL  AVERAGE  SCIENCE  NORMIT  MISSING 


CONDITIONING  VARIABLE  ID:  BACK0028 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  B009201A  (01 

002  B009201B  (02 

003  B009201C  (03 

004  B009201D  (04 

005  B009201M  (M 


TIMES  CHANGED  SCHLS  SINCE  1ST  GR;  NOT  PROMOTIONS? 
N04 


B009201 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 
1 ■ 

2 

3 OR  GREATER 
MISSING 


CONDITIONING  VARIABLE  ID:  BACK0031 


DESCRIPTION: 


GRADES /ASSESSMENTS : 

N04,  NOS, 

CONDITIONING 

VAR  LABEL: 

DISC0HOM 

NAEP  ID: 

B007401 

TYPE  OF  CONTRAST: 

CLASS 

001  DIS@H0M1 

(1 

) 000 

002  DIS0HOM2 

(2 

) 100 

003  DIS0HOM3 

(3 

) 010 

004  DIS@H0M4 

(4,M 

1 001 

CONDITIONING 

VARIABLE  ID: 

BACK0032 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 

N04,  NOS, 

CONDITIONING 
NAEP  ID: 

VAR  LABEL: 

B009301 

TYPE  OF  CONTRAST: 

CLASS 

001  DAILY 

(01  ] 

I 00000 

002  1-2  WEEK 

(02  ] 

1 10000 

003  1-2  MTH 

(03  ] 

f 01000 

004  NEVER 

(04  ) 

1 00100 

005  NO  COMP 

(05  ] 

1 00010 

006  MISSING 

(M  ] 

1 00001 

CONDITIONING 

VARIABLE  ID: 

BACK0033 

DESCRIPTION; 

ABOUT  HOW 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING 

VAR  LABEL: 

PGSREADl 

NAEP  ID: 

BOOllOl 

TYPE  OF  CONTRAST: 

CLASS 

001  PGS<6,? 

(5,M  ] 

f 0 

002  PGS>5 

(1,2, 3, 4 ] 

f 1 

CONDITIONING 

VARIABLE  ID: 

BACK0034 

DESCRIPTION; 

ABOUT  HOW 

grades /ASSESSMENTS : 

N12 

CONDITIONING 

VAR  LABEL: 

PGSREAD2 

NAEP  ID: 

BOOllOl 

TYPE  OF  CONTRAST: 

CLASS  • 

001  PGS<11,? 

(4,5,M  ) 

1 0 

HOW  OFTEN  DO  YOU  DISCUSS  THINGS  STUDIED  IN  SCHOOL  WITH  SOMEONE  AT  HOME? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT • CONTRASTS : 


DISCUSS  STUDIES  AT  HOME 
DISCUSS  STUDIES  AT  HOME 
DISCUSS  STUDIES  AT  HOME 
DISCUSS  STUDIES  AT  HOME 
MISSING 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MONTH 
NEVER  OR  HARDLY  EVER, 


SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  6 

NUMBER  OF  INDEPENDENT  CONTRASTS:  5 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
NO  COMPUTER  AT  HOME 
MISSING 


YOU  HAVE  TO  READ  FOR  SCHOOL  AND  HOMEWORK? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

PAGES  READ;  5 OR  FEWER  A DAY,  MISSING 
PAGES  READ:  6-10,  11-15,  16-20,  20  OR  MORE 


YOU  HAVE  TO  READ  FOR  SCHOOL  AND  HOMEWORK? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


PAGES  READ:  6-10,  5 OR  FEWER  A DAY,  MISSING 


O 

ERIC 
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002  PGS>10  (1,2,3  ) 1 


PAGES  READ:  11-15,  16-20,  20  OR  MORE 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0035 

WHICH  BEST  DESCRIBES  YOUR  HIGH-SCHOOL  PROGRAM? 

N12 

HS  PROG 

B008501  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CTASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001  GENERAL  (1  ) 0000 

002  ACADEMIC  (2  >1000 

003  VOC/TECH  (3  ) 0100 

004  OTHERPGM  (4  ) 0010 

005  HS  PGM-?  (M  ) 0001 


HIGH  SCHOOL  PROGRAM: 
HIGH  SCHOOL  PROGRAM: 
HIGH  SCHOOL  PROGRAM: 
HIGH  SCHOOL  PROGRAM: 
HIGH  SCHOOL  PROGRAM: 


GENERAL 

ACADEMIC /COLLEGE  PREP 
VOCATIONAL  OR  TECHNICAL 
OTHER 
MISSING 


CONDITIONING  VARIABLE  ID:  BACK0036 

DESCRIPTION:  SEMESTERS  ENGLISH/ LITERATURE/ WRITING  (MISSING  VS  NON-MISSING) 


GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL:  SEM  ENG 

NAEP  ID:  B007101 

TYPE  OF  CONTRAST:  CLASS 

001  SEMENG-?  (M  ) 0 

002  SEMENG-^  (1-9  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ENGLISH  SEMESTERS:  MISSING 

ENGLISH  SEMESTERS:  NOT  MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0037 

NUMBER  OF  SEMESTERS  ENGLISH/ LITERATURE /WRITING  (LINEAR) 

N12 

#ENG-LIN 

B007101  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


1 

1 


001  #ENG-LIN  (1-9,M=0  ) 0.0  + 1.0*X 

(LINEAR) 


NUMBER  OF  SEMESTERS  ENGLISH/LITERATURE/WRITING 


CONDITIONING  VARIABLE  ID:  BACK0038 

DESCRIPTION:  SEMESTERS  MATHEMATICS  (MISSING  VS  NON-MiSSiNG) 


GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL:  SEM  MAT 

NAEP  ID:  ' B007102 

TYPE  OF  CONTRAST:  CLASS 

001  SEMMAT-?  (M  ) 0 

002  SEMMAT-^  (1-9  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MATH  SEMESTERS:  MISSING 

MATH  SEMESTERS:  NOT  MISSING 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS:  . 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0039 

NUMBER  OF  SEMESTERS  MATHEMATICS  (LINEAR) 

N12 

#MAT-LIN 

B007102  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


1 

1 


001  #MAT-LIN  (1-9,M=0  ) 0.0  + 1.0*X 


NUMBER  OF  SEMESTERS  MATHEMATICS  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0040 

SEMESTERS  SCIENCE 
N12 

SEM  SCI 
B007103 
CLASS 


(MISSING  VS  NON-MISSING) 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  SEMSCI-?  (M  ) 0000 

002  SEMSCI-^  (1,2,3  ) 1000 

003  SEMSCI-^  (4,5  ) 0100 

004  SEMSCI-^  (6,7  ) 0010 

005  SEMSCI-^  (8,9  ) 0001 


SCIENCE  SEMESTERS 
SCIENCE  SEMESTERS 
SCIENCE  SEMESTERS 
SCIENCE  SEMESTERS 
SCIENCE  SEMESTERS 


MISSING 
ZERO  TO  2 
3 OR  4 
5 OR  6 
7 OR  MORE 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0041 

NUMBER  OF  SEMESTERS  SCIENCE  (LINEAR) 


N12 

#SCI-LIN 

B007103 

LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  #SCI-LIN  (1-9, M=0  ) 0.0  + 1.0*X 


NUMBER  OF  SEMESTERS  SCIENCE  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0042 

SEMESTERS  HISTORY/SOCIAL 
N12 

SEM  HIS 
B007104 
CLASS 


STUDIES/GEOGRAPHY  (MISSING  VS  NON-MISSING) 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  SEMHIS-?  (M  ) 0 

002  SEMHIS-^  (1-9  ) 1 


HISTORY  SEMESTERS:  MISSING 

HISTORY  SEMESTERS:  NOT  MISSING 


1 

1 


2 

1 


BEST  COPY  AVAILABLE 
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CONDITIONING  VARIABLE  ID:  BACK0043 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #HIS-LIN  (1-9, M=0 


NUMBER  OF  SEMESTERS  HI STORY/ SOCIAL  STUDIES/GEOGRAPHY  (LINEAR) 

N12 

#HIS-LIN 

B007104  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 0.0  + 1.0*X 


NUMBER  OF  SEMESTERS  HISTORY/ SOCIAL 
STUDIES/GEOGRAPHY  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /assessments : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SEMFLG-?  (M  ) 

002  SEMFLG-"'  (1-9  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  #FLG-LIN  (1-9,M=0 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SEMVOC-?  (M  ) 

002  SEMVOC-^'  (1-9  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0044 

SEMESTERS  FOREIGN  LANGUAGES  (MISSING  VS  NON-MISSING) 
N12 

SEM  FLG 
B007105 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


FOREIGN  LANGUAGE  SEMESTERS : 
FOREIGN  LANGUAGE  SEMESTERS: 


MISSING 
NOT  missing 


BACK0045 

NUMBER  OF  SEMESTERS  FOREIGN  LANGUAGES  (LINEAR) 

N12 

#FLG-LIN 

B007105  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


) 0.0  + 1.0*X 


NUMBER  OF  SEMESTERS  FOREIGN  LANGUAGES  (LINEAR) 


BACK0046 

SEMESTERS  VOCATIONAL/ TECHNICAL /BUSINESS  EDUCATION  (MISSING  VS  NON-MISSING) 
Nl2 

SEM  VOC 
B007106 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

VOC/TECH  SEMESTERS:  MISSING 

VOC/TECH  SEMESTERS:  NOT  MISSING 


BACK0047 

NUMBER  OF  SEMESTERS  VOCATIONAL/ TECHNICAL/ BUSINESS  EDUCATION  (LINEAR) 
N12 

#VOC-LIN 

B007106  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  #VOC-LIN  (1-9,M=0 


) 0.0  + 1.0*X 


NUMBER  OF  SEMESTERS  VOCATIONAL/ TECHNICAL/ BUSINESS 
EDUCATION  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

00 1 SEMART-?  (M  ) 

002  SEMART-''  (1-9  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0048 

SEMESTERS  ART/MUSiC  (MISSING  VS  NON-MISSING) 
N12 

SEM  ART 
B007107 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


ART/MUSIC  SEMESTERS: 
ART/MUSIC  SEMESTERS; 


MISSING 
NOT  MISSING 


BACK0049 

NUMBER  OF  SEMESTERS  ART/MUSIC  (LINEAR) 

N12 

#ART-LIN 

B007107  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  #ART-LIN  (1-9,M=0 


) 0.0  + 1.0*X 


NUMBER  OF  SEMESTERS  ART/MUSIC  (LINEAR) 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  SCITAKE 


SUBJOOOl 

WHAT  KIND  OF  SCI  CLASS  ARE  YOU  TAKING  THIS  YEAR*’ 
NOS,  SOS 


TYPE  OF  CONTRAST: 

CLASS 

001  NO  SCIEN 

(01  ] 

1 000000 

002  LIFESCI 

(02  i 

1 100000 

003  PHYSSCI 

(03  ] 

1 010000 

004  EATHSCI 

(04  ] 

1 001000 

005  GEN  SCI 

(05  ] 

1 000100 

006  intesci 

(06  i 

1 000010 

007  missing 

(M  i 

1 000001 

CONDITIONING 

VARIABLE  ID: 

BACK0050 

DESCRIPTION: 

INTERACTION: 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS, 

CONDITIONING 

VAR  LABEL: 

GEND/RAC 

NAEP  ID: 

N/A 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  TAKING  THIS  YEAR 
LIFE  SCIENCE 
PHYSICAL  SCIENCE 
EARTH  SCIENCE 
GENERAL  SCIENCE 

integrated  science 
missing 


GENDER  BY  RACE/ ETHNICITY 
N12  . 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 

INTERACTION 

001  G/R  11 

(11  ) 

010101 

002  G/R  12 

(12  ) 

-10000 

003  G/R  13 

(13  ) 

00-100 

004  G/R  14 

(14  ) 

0000-1 

005  G/R  21 

(21  ) 

-1-1-1 

006  G/R  22 

(22  ) 

010000 

007  G/R  23 

(23  ) 

000100 

OOS  G/R  24 

(24  ) 

000001 

CONDITIONING 

VARIABLE  ID: 

BACK0051 

DESCRIPTION: 

INTERACTION : GENDER 

grades /ASSESSMENTS : 

N04.'  N08,  S08,  N12 

CONDITIONING 

VAR  LABEL: 

GEND/TOL 

NAEP  ID: 

N/A 

TYPE  OF  CONTRAST: 

INTERACTION 

001  G/T  11 

(11  ) 

01010101 

002  G/T  12 

(12  ) 

1 -1000000 

003  G/T  13 

(13  ) 

' 00-10000 

004  G/T  14 

(14  ) 

1 0000-100 

005  G/T  15 

(15  ) 

1 000000-1 

006  G/T  21 

(21  ) 

1 -l-l-l-l 

007  G/T  22 

(22  ] 

1 01000000 

008  G/T  23 

(23  ] 

1 00010000 

009  G/T  24 

(24  1 

1 00000100 

010  G/T  25 

(25  1 

1 00000001 

CONDITIONING 

VARIABLE  ID: 

BACK0052 

DESCRIPTION: 

INTERACTION : GENDER 

GRADES /ASSESSMENTS : 

N04,  N08,  S08,  N12 

conditioning 

VAR  LABEL: 

GEND/PAR 

NAEP  ID: 

N/A 

TYPE  OF  CONTRAST: 

INTERACTION 

001  G/P  11 

(11 

) 01010101 

002  G/P  12 

(12 

) -1000000 

003  G/P  13 

(13 

) 00-10000 

004  G/P  14 

(14 

) 0000-100 

005  G/P  15 

(15 

) 000000-1 

006  G/P  21 

(21 

) -l-l-l-l 

007  G/P  22 

(22 

) 01000000 

008  G/P  23 

(23 

) 00010000 

009  G/P  24 

(24 

) 00000100 

010  G/P  25 

(25 

) 00000001 

CONDITIONING 

VARIABLE  ID: 

BACK0053 

DESCRIPTION: 

INTERACTION:  GENDER 

GRADES/ASSESSMENTS : 

N04,  N08,  S08,  N12 

CONDITIONING 

VAR  LABEL: 

GEND/SCH 

NAEP  ID: 

N/A 

TYPE  OF  CONTRAST: 

INTERACTION 

001  G/S  11 

(11 

) 0101 

002  G/S  12 

(12 

) -100 

003  G/S  13 

(13 

) 00-1 

004  G/S  21 

(21 

) -1-1 

005  G/S  22 

(22 

) 0100 

006  G/S  23 

(23 

) 0001 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

GEND/RAC  INTACT:  1.  MALE  1.  WHI/AI/0 

GEND/RAC  INTACT:  1.  MALE  2.  BLACK 

GEND/RAC  INTACT:  1.  MALE  3.  HISPANIC 

GEND/RAC  INTACT:  1.  MALE  4.  ASIAN 

GEND/RAC  INTACT:  2.  FEMALE  1.  WHI/Al/0 

GEND/RAC  INTACT:  2.  FEMALE  2.  BLACK 

GEND/RAC  INTACT:  2.  FEMALE  3.  HISPANIC 

GEND/RAC  INTACT:  2.  FEMALE  4.  ASIAN 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


10 

4 


GEND/TOL 

INTACT : 

1. 

MALE 

.1. 

BIG 

CTY5 

GEND/TOL 

INTACT: 

1. 

MALE 

2 . 

MID 

CTY5 

GEND/TOL 

INTACT: 

1 . 

MALE 

3 . 

FR/BTWN5 

GEND/TOL 

INTACT : 

1. 

MALE 

4. 

SML 

TWN5 

GEND/TOL 

INTACT : 

1. 

MALE 

5. 

RURAL 5 

GEND/TOL 

INTACT : 

2 . 

FEMALE 

1. 

BIG 

CTY5 

GEND/TOL 

INTACT: 

2 . 

FEMALE 

2 . 

MID 

CTY5 

GEND/TOL 

INTACT: 

2 . 

FEMALE 

3 . 

FR/BTWN5 

GEND/TOL 

. INTACT : 

2 . 

FEMALE 

4. 

SML 

TWN5 

GEND/TOL 

INTACT : 

2 . 

FEMALE 

5. 

RURAL 5 

EDUCATION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  10 

NUMBER  OF  INDEPENDENT  CONTRASTS:  4 


GEND/PAR 
GEND/PAR 
GEND/PAR 
GEND/PAR 
GEND/ PAR 
GEND/PAR 
GEND/PAR 
GEND/PAR 
GEND/PAR 
GEND/PAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  6 

NUMBER  OF  INDEPENDENT  CONTRASTS:  2 


INTACT: 

1. 

MALE 

1. 

< HS 

INTACT : 

1. 

MALE 

2 . 

HS  GRAD 

INTACT : 

1. 

MALE 

3 . 

POST  HS 

INTACT : 

1. 

MALE 

4. 

COL  GRAD 

INTACT : 

1. 

MALE 

5. 

PARED-? 

INTACT : 

2 . 

FEMALE 

1. 

< HS 

INTACT : 

2 . 

FEMALE 

2. 

HS  GRAD 

INTACT : 

2 . 

FEMALE 

3. 

POST  HS 

INTACT : 

2 . 

FEMALE 

4. 

COL  GRAD 

INTACT : 

2 . 

FEMALE 

5. 

PARED-? 

INTACT 

1. 

MALE 

1. 

PUBLIC 

INTACT 

1. 

MALE 

2 . 

PRIVATE 

INTACT 

1. 

MALE 

3 . 

CATHOLIC 

INTACT 

2. 

FEMALE 

1. 

PUBLIC 

INTACT 

2. 

FEMALE 

2. 

PRIVATE 

INTACT 

2. 

FEMALE 

3 . 

CATHOLIC 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0054 

INTERACTION:  RACE/ETHNICITY  BY  TYPE  OF  LOCALE  (5  CATEGORIES) 

N04,  NOS.  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


N/A 

INTERACTION 


20 

12 


001 

R/T 

11 

(11 

002 

R/T 

12 

(12 

003 

R/T 

13 

(13 

004 

R/T 

14 

(14 

005 

R/T 

15 

(15 

006 

R/T 

21 

(21 

007 

R/T 

22 

(22 

008 

R/T 

23 

(23 

009 

R/T 

24 

(24 

010 

R/T 

25 

(25 

Oil 

R/T 

31 

(31 

012 

R/T 

32 

(32 

013 

R/T 

33 

(33 

014 

R/T 

34 

(34 

015 

R/T 

35 

(35 

016 

R/T 

41 

(41 

017 

R/T 

42 

(42 

018 

R/T 

43 

(43 

019 

R/T 

44 

(44 

) 010101010101010101010101 
) -1000000-1000000-1000000 
) 00-1000000-1000000-10000 
) 0000-1000000-1000000-100 
) 000000-1000000-1000000-1 
) -1-1-1-10000000000000000 
) 010000000000000000000000 
) 000100000000000000000000 
) 000001000000000000000000 
) 000000010000000000000000 
) 00000000-1-1-1-100000000 
) 000000000100000000000000 
) 000000000001000000000000 
) 000000000000010000000000 
) 000000000000000100000000 
) 0000000000000000-1-1-1-1 
) 000000000000000001000000 
) 000000000000000000010000 
) 000000000000000000000100 


RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 
RACE/TOL  INTACT: 


1.  WHI/AI/0 

1.  WHI/AI/0 

1.  WHI/AI/0 

1.  WHI/AI/0 

1.  WHI/AI/0 

2 . BLACK 

2.  BLACK 

2.  BLACK 

2 .  BLACK 

2 . BLACK 

3.  HISPANIC 

3.  HISPANIC 

3.  HISPANIC 

3.  HISPANIC 

3.  HISPANIC 

4.  ASIAN 

4.  ASIAN 

4.  ASIAN 

4.  ASIAN 


1.  BIG  CTY5 

2 . MID  CTY5 

3.  FR/BTWN5 

4.  SML  TWN5 

5.  RURAL5 

1.  BIG  CTY5 

2 . MID  CTY5 

3.  FR/BTWN5 

4 . SML  TWN5 

5 . RURAL5 

1.  BIG  CTY5 

2 . MID  CTY5 

3.  FR/BTWN5 

4 . SML  TWN5 

5.  RURAL5 

1.  BIG  CTY5 

2.  MID  CTY5 

3.  FR/BTWN5 

4.  SML  TWN5 


BEST  COPY  AVAILABLE 


555 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


020  R/T  45 

(45 

) 000000000000000000000001 

CONDITIONING 

! VARIABLE  ID: 

BACK0055 

DESCRIPTION: 

INTERACTION:  RACE/ETHNICITY 

GRADES/ASSESSMENTS : 

N04,  NOS,  SOS,  N12 

CONDITIONING 

1 VAR  LABEL: 

RACE/ PAR 

NAEP  ID: 

N/A 

TYPE  OF  CONTRAST: 

INTERACTION 

001  R/P  11 

(11 

) 010101010101010101010101 

002  R/P  12 

(12 

) -1000000-1000000-1000000 

003  R/P  13 

(13 

) 00-1000000-1000000-10000 

004  R/P  14 

(14 

) 0000-1000000-1000000-100 

005  R/P  15 

(15 

) 000000-1000000-1000000-1 

006  R/P  21 

(21 

) -1-1-1-10000000000000000 

007  R/P  22 

(22 

) 0100000000000.00000000000 

OOS  R/P  23 

(23 

) 000100000000000000000000 

009  R/P  24 

(24 

) 000001000000000000000000 

010  R/P  25 

(25 

) 000000010000000000000000 

Oil  R/P  31 

(31 

) 00000000-1-1-1-100000000 

012  R/P  32 

(32 

) 000000000100000000000000 

013  R/P  33 

(33 

) 000000000001000000000000 

014  R/P  34 

(34  1 

1 000000000000010000000000 

015  R/P  35 

(35  1 

1 000000000000000100000000 

016  R/P  41 

(41  1 

1 0000000000000000-1-1-1-1 

017  R/P  42 

(42  1 

1 000000000000000001000000 

OIS  R/P  43 

(43  1 

1 000000000000000000010000 

019  R/P  44 

(44  ) 

1 000000000000000000000100 

020  R/P  45 

(45  ) 

1 000000000000000000000001 

CONDITIONING 

VARIABLE  ID: 

BACK0056 

DESCRIPTION: 

INTERACTION:  RACE/ETHNICITY 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS,  N12 

CONDITIONING 

VAR  LABEL: 

RACE/SCH 

NAEP  ID: 

N/A 

TYPE  OF  CONTRAST: 

INTERACTION 

001  R/S  11 

(11  ) 

010101010101 

002  R/S  12 

(12  ) 

-100-100-100 

003  R/S  13 

(13  ) 

00-100-100-1 

004  R/S  21 

(21  ) 

-1-100.000000 

005  R/S  22 

(22  ) 

010000000000 

006  R/S  23 

(23  ) 

000100000000 

007  R/S  31 

(31  ) 

0000-1-10000 

OOS  R/S  32 

(32  ) 

000001000000 

009  R/S  33 

(33  ) 

000000010000 

010  R/S  41 

(41  ) 

00000000-1-1 

Oil  R/S  42 

(42  ) 

000000000100 

012  R/S  43 

(43  ) 

000000000001 

CONDITIONING 

VARIABLE  ID: 

BACK0057 

DESCRIPTION: 

INTERACTION:  TYPE  OF  LOCALE 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS,  N12 

CONDITIONING 

VAR  LABEL: 

TOL5/PAR 

NAEP  ID: 

N/A 

RACE/TOL  INTACT:  4.  ASIAN  5.  RURAL5 


BY  PARENTS ' EDUCATION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE /PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE /PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 
RACE/ PAR 


20 

12 


INTACT : 

1.  WHI/AI/0 

1. 

< HS 

INTACT : 

1.  WHI/AI/0 

2 . 

HS  GRAD 

INTACT : 

1.  WHI/AI/0 

3. 

POST  HS 

INTACT : 

1.  WHI/AI/0 

4. 

COL  GRAD 

INTACT : 

1.  WHI/AI/0 

5. 

PARED-? 

INTACT : 

2.  BLACK 

1. 

< HS 

INTACT : 

2.  BLACK 

2 . 

HS  GRAD 

INTACT : 

2.  BLACK 

3. 

POST  HS 

INTACT : 

2.  BLACK 

4. 

COL  GRAD 

INTACT : 

2.  BLACK 

5. 

PARED- ? 

INTACT : 

3.  HISPANIC 

1. 

< HS 

INTACT : 

3.  HISPANIC 

2. 

HS  GRAD 

INTACT : 

3.  HISPANIC 

3. 

POST  HS 

INTACT : 

3.  HISPANIC 

4. 

COL  GRAD 

INTACT : 

3.  HISPANIC 

5. 

PARED- ? 

INTACT : 

4.  ASIAN 

1. 

< HS 

INTACT : 

4.  ASIAN 

2. 

HS  GRAD 

INTACT : 

4.  ASIAN 

3. 

POST  HS 

INTACT : 

4.  ASIAN 

4. 

COL  GRAD 

INTACT : 

4.  ASIAN 

5. 

PARED-? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 

RACE/SCH 


12 

6 


INTACT : 

1.  WHI/AI/0 

1. 

PUBLIC 

INTACT : 

1.  WHI/AI/0 

2. 

PRIVATE 

INTACT : 

1.  WHI/AI/0 

3. 

CATHOLIC 

INTACT : 

2.  BLACK 

1. 

PUBLIC 

INTACT : 

2.  BLACK 

2 . 

PRIVATE 

INTACT : 

2.  BLACK 

3. 

CATHOLIC 

INTACT : 

3.  HISPANIC 

1. 

PUBLIC 

INTACT : 

3.  HISPANIC 

2. 

PRIVATE 

INTACT : 

3 . HISPANIC 

3. 

CATHOLIC 

INTACT : 

4.  ASIAN 

1. 

PUBLIC 

INTACT : 

4.  ASIAN 

2. 

PRIVATE 

INTACT : 

4.  ASIAN 

3. 

CATHOLIC 

TYPE  OF 

CONTRAST 

001 

T/P 

11 

(11 

002 

T/P 

12 

(12 

003 

T/P 

13 

(13 

004 

T/P 

14 

(14 

005 

T/P 

15 

(15 

006 

T/P 

21 

(21 

007 

T/P 

22 

(22 

OOS 

T/P 

23 

(23 

009 

T/P 

24 

(24 

010 

T/P 

25 

(25 

Oil 

T/P 

31 

(31 

012 

T/P 

32 

(32 

013 

T/P 

33 

(33 

014 

T/P 

34 

(34 

015 

T/P 

35 

(35 

016 

T/P 

41 

(41 

017 

T/P 

42 

(42 

OIS 

T/P 

43 

(43 

019 

T/P 

44 

(44 

020 

T/P 

45 

(45 

021 

T/P 

51 

(51 

022 

T/P 

52 

(52 

023 

T/P 

53 

(53 

024 

T/P 

54 

(54 

025 

T/P 

55 

(55 

INTERACTION 

) 01010101010101010101010101010101  TOL5/PAR 
) -1000000-1000000-1000000-1000000  TOL5/PAR 
) 00-1000000-1000000-1000000-10000  TOL5/PAR 
) 0000-1000000-1000000-1000000-100  TOL5/PAR 
) 000000-1000000-1000000-1000000-1  TOL5/PAR 
) -1-1-1-1000000000000000000000000  TOL5/PAR 
) 01000000000000000000000000000000  TOL5/PAR 
) 00010000000000000000000000000000  TOL5/PAR 
) 00000100000000000000000000000000  TOL5/PAR 
) 00000001000000000000000000000000  TOL5/PAR 
) 00000000-1-1-1-10000000000000000  TOL5/PAR 
) 00000000010000000000000000000000  TOL5/PAR 
) 00000000000100000000000000000000  TOL5/PAR 
) 00000000000001000000000000000000  TOL5/PAR 
) 00000000000000010000000000000000  TOL5/PAR 
) 0000000000000000-1-1-1-100000000  TOL5/PAR 
) 00000000000000000100000000000000  TOL5/PAR 
) 00000000000000000001000000000000  TOL5/PAR 
) 00000000000000000000010000000000  TOL5/PAR 
) 00000000000000000000000100000000  TOL5/PAR 
) 000000000000000000000000-1-1-1-1  TOL5/PAR 
) 00000000000000000000000001000000  TOL5/PAR 
) 00000000000000000000000000010000  TOL5/PAR 
) 00000000000000000000000000000100  TOL5/PAR 
) 00000000000000000000000000000001  TOL5/PAR 


'S  EDUCATION 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS : 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


25 

16 


INTACT : 

1. 

BIG 

CTY5 

1. 

< HS 

INTACT : 

1. 

BIG 

CTY5 

2 . 

HS  GRAD 

INTACT : 

1. 

BIG 

CTY5 

3. 

POST  HS 

INTACT : 

1. 

BIG 

CTY5 

4. 

COL  GRAD 

INTACT : 

1. 

BIG 

CTY5 

5. 

PARED- ? 

INTACT : 

2. 

MID 

CTY5 

1. 

< HS 

INTACT : 

2. 

MID 

CTY5 

2. 

HS  GRAD 

INTACT : 

2 . 

MID 

CTY5 

3. 

POST  HS 

INTACT : 

2 . 

MID 

CTY5 

4. 

COL  GRAD 

INTACT : 

2 . 

MID 

CTY5 

5. 

PARED-? 

INTACT : 

3. 

FR/BTWN5 

1. 

< HS 

INTACT : 

3. 

FR/BTWN5 

2. 

HS  GRAD 

INTACT : 

3. 

FR/BTWN5 

3. 

POST  HS 

INTACT : 

3. 

FR/BTWN5 

4. 

COL  GRAD 

INTACT : 

3. 

FR/BTWN5 

5. 

PARED-? 

INTACT : 

4. 

SML 

TWN5 

1. 

< HS 

INTACT : 

4. 

SML 

TWN5 

2 . 

HS  GRAD 

INTACT : 

4. 

SML 

TWN5 

3 . 

POST  HS 

INTACT : 

4. 

SML 

TWN5 

4. 

COL  GRAD 

INTACT : 

4. 

SML 

TWN5 

5. 

PARED- ? 

INTACT : 

5. 

RURAL5 

1. 

< HS 

INTACT: 

5. 

RURAL5 

2. 

HS  GRAD 

INTACT : 

5. 

RURAL5 

3. 

POST  HS 

INTACT : 

5. 

RURAL5 

4. 

COL  GRAD 

INTACT : 

5. 

RURAL5 

5. 

PARED- ? 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

grades/assessments : 

CONDITIONING  VAR  LABEL: 


BACK0058 

INTERACTION:  TYPE  OF  LOCALE  (5  CATEGORIES)  BY  SCHOOL  TYPE 

N04,  NOS.  SOS,  N12 

TOL5/SCH 


ERIC 


fin7 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


NAEP  ID: 

N/A 

TYPE  OF 

CONTRAST : 

INTERACTION 

001 

T/S 

11 

(11 

) 0101010101010101 

002 

T/S 

12 

(12 

) -100-100-100-100 

003 

T/S 

13 

(13 

) 00-100-100-100-1 

004 

T/S 

21 

(21 

) -1-1000000000000 

005 

T/S 

22 

(22 

) 0100000000000000 

006 

T/S 

23 

(23 

) 0001000000000000 

007 

T/S 

31 

(31 

) 0000-1-100000000 

008 

T/S 

32 

(32 

) 0000010000000000 

009 

T/S 

33 

(33 

) 0000000100000000 

010 

T/S 

41 

(41 

) 00000000-1-10000 

Oil 

T/S 

42 

(42 

) 0000000001000000 

012 

T/S 

43 

(43 

) 0000000000010000 

013 

T/S 

51 

(51 

) 000000000000-1-1 

014 

T/S 

52 

(52 

) 0000000000000100 

015 

T/S 

53 

(53 

) 0000000000000001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 

TOL5/SCH 


15 

8 


INTACT 

1. 

BIG  CTY5 

1. 

PUBLIC 

INTACT 

1. 

BIG  CTY5 

2 . 

PRIVATE 

INTACT 

1. 

BIG  CTY5 

3 . 

CATHOLIC 

INTACT 

2 . 

MID  CTY5 

1. 

PUBLIC 

INTACT 

2 . 

MID  CTY5 

2 . 

PRIVATE 

INTACT 

2 . 

MID  CTY5 

3. 

CATHOLIC 

INTACT 

3 . 

FR/BTWN5 

1. 

PUBLIC 

INTACT 

3 . 

FR/BTWN5 

2 . 

PRIVATE 

INTACT 

3 . 

FR/BTWN5 

3. 

CATHOLIC 

INTACT 

4 . 

SML  TWN5 

1. 

PUBLIC 

INTACT 

4. 

SML  TWN5 

2 . 

PRIVATE 

INTACT 

4. 

SML  TWN5 

3 . 

CATHOLIC 

INTACT 

5. 

RURAL5 

1. 

PUBLIC 

INTACT 

5. 

RURAL5 

2 . 

PRIVATE 

INTACT 

5. 

RURAL5 

3 . 

CATHOLIC 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


BACK0059 

INTERACTION : PARENTS ' 

N04,  NOS,  SOS,  N12 
PARE/SCH 


EDUCATION  BY  SCHOOL  TYPE 


NAEP 

ID: 

N/A 

TYPE 

: OF 

CONTRAST : 

INTERACTION 

001 

P/S 

11 

(11  ) 

1 0101010101010101 

002 

P/S 

12 

(12  ) 

t -100-100-100-100 

003 

P/S 

13 

(13  ) 

1 00-100-100-100-1 

004 

P/S 

21 

(21  ) 

1 -1-1000000000000 

005 

P/S 

22 

(22  ] 

1 0100000000000000 

006 

P/S 

23 

(23  ) 

t 0001000000000000 

007 

P/S 

31 

(31  ] 

1 0000-1-100000000 

008 

P/S 

32 

(32  ] 

1 0000010000000000 

009 

P/S 

33 

(33  ] 

1 0000000100000000 

010 

P/S 

41 

(41  ] 

1 00000000-1-10000 

Oil 

P/S 

42 

(42  ] 

1 0000000001000000 

012 

P/S 

43 

(43  ; 

1 0000000000010000 

013 

P/S 

51 

(51  : 

1 000000000000-1-1 

014 

P/S 

52 

(52  ; 

1 0000000000000100 

015 

P/S 

53 

(53  ; 

1 0000000000000001 

CONDITIONING 

VARIABLE  ID: 

BACK 00 60 

DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


INTERACTION: 
NOS,  SOS 
GEND/ 

N/A 

INTERACTION 


001 

G/ 

11 

(11 

) 010101010101 

002 

G/ 

12 

(12 

) -10000000000 

003 

G/ 

13 

(13 

) 00-100000000 

004 

G/ 

14 

(14 

) 0000-1000000 

005 

G/ 

15 

(15 

) 000000-10000 

006 

G/ 

16 

(16 

) 00000000-100 

007 

G/ 

17 

(17 

) 0000000000-1 

008 

G/ 

21 

(21 

) -l-l-l-l-l-l 

009 

G/ 

22 

(22 

) 010000000000 

010 

G/ 

23 

(23 

) 000100000000 

oil 

G/ 

24 

(24 

) 000001000000 

012 

G/ 

25 

(25 

) 000000010000 

013 

G/ 

26 

(26 

) 000000000100 

014 

G/ 

27 

(27 

) 000000000001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  15 

NUMBER  OF  INDEPENDENT  CONTRASTS:  8 


PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/ SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 
PARE/SCH 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  14 

NUMBER  OF  INDEPENDENT  CONTRASTS:  6 


GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 

GEND/ 


INTACT: 

1 . 

< HS 

1. 

PUBLIC 

INTACT : 

1 . 

< HS 

2 . 

PRIVATE 

INTACT : 

1. 

< HS 

3. 

CATHOLIC 

INTACT : 

2 . 

HS  GRAD 

1. 

PUBLIC 

INTACT : 

2 . 

HS  GRAD 

2. 

PRIVATE 

INTACT : 

2 . 

HS  GRAD 

3 . 

CATHOLIC 

INTACT : 

3 . 

POST  HS 

1. 

PUBLIC 

INTACT : 

3 . 

POST  HS 

2 . 

PRIVATE 

INTACT : 

3 . 

POST  HS 

3. 

CATHOLIC 

INTACT : 

4. 

COL  GRAD 

1. 

PUBLIC 

INTACT : 

4 . 

, COL  GRAD 

2 . 

PRIVATE 

INTACT : 

4 . 

, COL  GRAD 

3. 

CATHOLIC 

INTACT : 

5. 

, PARED- ? 

1. 

PUBLIC 

INTACT : 

5. 

, PARED- ? 

2 . 

PRIVATE 

INTACT ; 

5 , 

, PARED-? 

3. 

CATHOLIC 

MCING  THIS 

YEAR 

INTACT : 

1. 

MALE 

1. 

NO  SCIEN 

INTACT : 

1. 

MALE 

2 . 

LIFESCI 

INTACT ; 

1. 

MALE 

3. 

PHYSSCI 

INTACT : 

1. 

MALE 

4. 

EATHSCI 

INTACT : 

1. 

MALE 

5. 

GEN  SCI 

INTACT : 

1. 

MALE 

6. 

INTESCI 

INTACT : 

1. 

MALE 

7 . 

MISSING 

INTACT : 

2. 

FEMALE 

1. 

NO  SCIEN 

INTACT : 

2. 

FEMALE 

2 . 

LIFESCI 

INTACT : 

2. 

FEMALE 

3 . 

PHYSSCI 

INTACT : 

2 . 

FEMALE 

4 . 

EATHSCI 

INTACT: 

2 . 

FEMALE 

5. 

GEN  SCI 

INTACT : 

2 . 

FEMALE 

6. 

INTESCI 

INTACT : 

2 . 

FEMALE 

7. 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0062 
INTERACTION: 
NOS,  SOS 
RACE/ 

N/A 

INTERACTION 


001 

R/ 

11 

(11 

) 333333333333333333 

002 

R/ 

12 

(12 

) 122222122222122222 

003 

R/ 

13 

(13 

) 212222212222212222 

004 

R/ 

14 

(14 

) 221222221222221222 

005 

R/ 

15 

(15 

) 222122222122222122 

006 

R/ 

16 

(16 

) 222212222212222212 

007 

R/ 

17 

(17 

) 222221222221222221 

008 

R/ 

21 

(21 

) 111111222222222222 

009 

R/ 

22 

(22 

) 322222222222222222 

010 

R/ 

23 

(23 

) 232222222222222222 

Oil 

R/ 

24 

(24 

) 223222222222222222 

012 

R/ 

25 

(25 

) 222322222222222222 

013 

R/ 

26 

(26 

) 222232222222222222 

014 

R/ 

27 

(27 

) 222223222222222222 

015 

R/ 

31 

(31 

) 222222111111222222 

016 

R/ 

32 

(32 

) 222222322222222222 

RACE/ETHNICITY  BY  SCIENCE  COURSES  TAKING  THIS  YEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 

RACE/ 


28 

18 


INTACT : 

1. 

WHI/AI/0 

1. 

NO  SCIEN 

INTACT : 

1. 

WHI/AI/0 

2 . 

LIFESCI 

INTACT: 

1. 

WHI/AI/0 

3 . 

PHYSSCI 

INTACT : 

1. 

WHI/AI/0 

4 . 

EATHSCI 

INTACT : 

1. 

WHI/AI/0 

5 . 

GEN  SCI 

INTACT: 

1. 

WHI/AI/0 

6. 

INTESCI 

INTACT : 

1. 

WHI/AI/0 

7 . 

MISSING 

INTACT : 

2. 

BLACK 

1. 

NO  SCIEN 

INTACT: 

2 . 

BLACK 

2 . 

LIFESCI 

INTACT: 

2 . 

BLACK 

3 . 

PHYSSCI 

INTACT : 

2 . 

BLACK 

4 . 

EATHSCI 

INTACT : 

2 . 

BLACK 

5. 

GEN  SCI 

INTACT: 

2 . 

BLACK 

6. 

INTESCI 

INTACT : 

2 . 

BLACK 

7 . 

MISSING 

INTACT : 

3. 

HISPANIC 

1. 

NO  SCIEN 

INTACT: 

3. 

HISPANIC 

2. 

LIFESCI 

ERIC 


6u8 


557 


BEST  COPY  AVAILABLE 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


017 

R/ 

33 

(33 

018 

R/ 

34 

(34 

019 

R/ 

35 

(35 

020 

R/ 

36 

(36 

021 

R/ 

37 

(37 

022 

R/ 

41 

(41 

023 

R/ 

42 

(42 

024 

R/ 

43 

(43 

025 

R/ 

44 

(44 

026 

R/ 

45 

(45 

027 

R/ 

46 

(46 

028 

R/ 

47 

(47 

CONDITIONING  VARIABLE  ID 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 222222232222222222 
) 222222223222222222 
) 222222222322222222 
) 222222222232222222 
) 222222222223222222 
) 222222222222111111 
) 222222222222322222 
) 222222222222232222 
) 222222222222223222 
) 222222222222222322 
) 222222222222222232 
) 222222222222222223 

BACK0064 

INTERACT I ON : PARENTS ' 

NOS,  SOS 

PARE/ 

N/A 

INTERACTION 


RACE/  INTACT:  3.  HISPANIC  3.  PHYSSCI 

RACE/  INTACT:  3.  HISPANIC  4.  EATHSCI 

RACE/  INTACT:  3.  HISPANIC  5.  GEN  SCI 

RACE/  INTACT:  3.  HISPANIC  6.  INTESCI 

RACE/  INTACT:  3.  HISPANIC  7.  MISSING 

RACE/  INTACT:  4.  ASIAN  1.  NO  SCIEN 

RACE/  INTACT:  4.  ASIAN  2.  LIFESCI 

RACE/  INTACT:  4.  ASIAN  3.  PHYSSCI 

RACE/  INTACT:  4.  ASIAN  4.  EATHSCI 

RACE/  INTACT:  4.  ASIAN  5.  GEN  SCI 

race/  INTACT:  4.  ASIAN  6.  INTESCI 

RACE/  INTACT:  4.  ASIAN  7.  MISSING 


EDUCATION  BY  SCIENCE  COURSES  TAKING  THIS  YEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  3 

NUMBER  OF  INDEPENDENT  CONTRASTS:  2 


001 

P/ 

11 

(11 

002 

P/ 

12 

(12 

003 

P/ 

13 

(13 

004 

P/ 

14 

(14 

005 

P/ 

15 

(15 

006 

P/ 

16 

(16 

007 

P/ 

17 

(17 

008 

P/ 

21 

(21 

009 

P/ 

22 

(22 

010 

P/ 

23 

(23 

Oil 

P/ 

24 

(24 

012 

P/ 

25 

(25 

013 

P/ 

26 

(26 

014 

P/ 

27 

(27 

015 

P/ 

31 

(31 

016 

P/ 

32 

(32 

017 

P/ 

33 

(33 

018 

P/ 

34 

(34 

019 

P/ 

35 

(35 

020 

P/ 

36 

(36 

021 

P/ 

37 

(37 

022 

P/ 

41 

(41 

023 

P/ 

42 

(42 

024 

P/ 

43 

(43 

025 

P/ 

44 

(44 

02  6 

P/ 

45 

(45 

027 

P/ 

46 

(46 

028 

P/ 

47 

(47 

029 

P/ 

51 

(51 

030 

P/ 

52 

(52 

031 

P/ 

53 

(53 

032 

P/ 

54 

(54 

033 

P/ 

55 

(55 

034 

P/ 

56 

(56 

035 

P/ 

57 

(57 

) 333333333333333333333333 
) 122222122222122222122222 
) 212222212222212222212222 
) 221222221222221222221222 
) 222122222122222122222122 
) 222212222212222212222212 
) 222221222221222221222221 
) 111111222222222222222222 
) 322222222222222222222222 
) 232222222222222222222222 
) 223222222222222222222222 
) 222322222222222222222222 
) 222232222222222222222222 
) 222223222222222222222222 
) 222222111111222222222222 
) 222222322222222222222222 
) 222222232222222222222222 
) 222222223222222222222222 
) 222222222322222222222222 
) 222222222232222222222222 
) 222222222223222222222222 
) 222222222222111111222222 
) 222222222222322222222222 
) 222222222222232222222222 
) 222222222222223222222222 
) 222222222222222322222222 
) 222222222222222232222222 
) 222222222222222223222222 
) 222222222222222222111111 
) 222222222222222222322222 
) 222222222222222222232222 
) 222222222222222222223222 
) 222222222222222222222322 
) 222222222222222222222232 
) 222222222222222222222223 


PARE/ 

INTACT : 

1 1 

PARE/ 

INTACT ; 

: 1 

PARE/ 

INTACT : 

: 1 

PARE/ 

INTACT : 

1 

PARE/ 

INTACT : 

1 

PARE/ 

INTACT : 

1 

PARE/ 

INTACT : 

1 

PARE/ 

INTACT : 

2 

PARE/ 

INTACT : 

2. 

PARE/ 

INTACT : 

2 . 

PARE/ 

INTACT : 

2 , 

PARE/ 

INTACT : 

2 . 

PARE/ 

INTACT : 

2 . 

PARE/ 

INTACT : 

2 . 

PARE/ 

INTACT : 

3 . 

PARE/ 

INTACT : 

3 . 

PARE/ 

INTACT: 

3 . 

PARE/ 

INTACT : 

3 . 

PARE/ 

INTACT: 

3 . 

PARE/ 

INTACT : 

3 . 

PARE/ 

INTACT: 

3 . 

PARE/ 

INTACT : 

4 . 

PARE/ 

INTACT: 

4. 

PARE/ 

INTACT : 

4. 

PARE/ 

INTACT : 

4. 

PARE/ 

INTACT : 

4. 

PARE/ 

INTACT : 

4. 

PARE/ 

INTACT : 

4. 

PARE/ 

INTACT : 

5. 

PARE/ 

INTACT : 

5. 

PARE/ 

INTACT : 

5. 

PARE/ 

INTACT : 

5. 

PARE/ 

INTACT : 

5. 

PARE/ 

INTACT : 

5. 

PARE/ 

INTACT : 

5. 

< HS 

1 

. NO  SCIEN 

< HS 

2 . 

. LIFESCI 

< HS 

3 , 

. PHYSSCI 

< HS 

4, 

. EATHSCI 

< HS 

5. 

GEN  SCI 

< HS 

6. 

INTESCI 

< HS 

7. 

MISSING 

HS  GRAD 

1. 

NO  SCIEN 

HS  GRAD 

2 . 

LIFESCI 

HS  GRAD 

3 . 

PHYSSCI 

HS  GRAD 

4. 

EATHSCI 

HS  GRAD 

5. 

GEN  SCI 

HS  GRAD 

6. 

INTESCI 

HS  GRAD 

7. 

MISSING 

POST  HS 

1. 

NO  SCIEN 

POST  HS 

2 . 

LIFESCI 

POST  HS 

3 . 

PHYSSCI 

POST  HS 

4. 

EATHSCI 

POST  HS 

5. 

GEN  SCI 

POST  HS 

6. 

INTESCI 

POST  HS 

7. 

MISSING 

COL  GRAD 

1. 

NO  SCIEN 

COL  GRAD 

2 . 

LIFESCI 

COL  GRAD 

3 . 

PHYSSCI 

COL  GRAD 

4. 

EATHSCI 

COL  GRAD 

5. 

GEN  SCI 

COL  GRAD 

6. 

INTESCI 

COL  GRAD 

7 . 

MISSING 

PARED-? 

1. 

NO  SCIEN 

PARED-? 

2 . 

LIFESCI 

PARED-? 

3 . 

PHYSSCI 

PARED-? 

4. 

EATHSCI 

PARED-? 

5. 

GEN  SCI 

PARED-? 

6. 

INTESCI 

PARED- ? 

7. 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001 

T/ 

11 

(11 

002 

T/ 

12 

(12 

003 

T/ 

13 

(13 

004 

T/ 

14 

(14 

005 

T/ 

15 

(15 

006 

T/ 

16 

(16 

007 

T/ 

17 

(17 

008 

T/ 

21 

(21 

009 

T/ 

22 

(22 

010 

T/ 

23 

(23 

Oil 

T/ 

24 

(24 

012 

T/ 

25 

(25 

013 

T/ 

26 

(26 

014 

T/ 

27 

(27 

015 

T/ 

31 

(31 

016 

T/ 

32 

(32 

017 

T/ 

33 

(33 

018 

T/ 

34 

(34 

019 

T/ 

35 

(35 

020 

T/ 

36 

(36 

021 

T/ 

37 

(37 

022 

T/ 

41 

(41 

023 

T/ 

42 

(42 

024 

T/ 

43 

(43 

BACK0066 
INTERACTION: 
N08,  S08 
TOL5/ 

N/A 

INTERACTION 


TYPE  OF  LOCALE  (5  CATEGORIES)  BY  SCIENCE  COURSES  TAKING  THIS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


333333333333333333333333 

122222122222122222122222 

212222212222212222212222 

221222221222221222221222 

222122222122222122222122 

222212222212222212222212 

222221222221222221222221 

111111222222222222222222 

322222222222222222222222 

232222222222222222222222 

223222222222222222222222 

222322222222222222222222 

222232222222222222222222 

222223222222222222222222 

222222111111222222222222 

222222322222222222222222 

222222232222222222222222 

222222223222222222222222 

222222222322222222222222 

222222222232222222222222 

222222222223222222222222 

222222222222111111222222 

222222222222322222222222 

222222222222232222222222 


TOL5/ 

INTACT : 

1 

TOL5/ 

INTACT : 

1 

TOL5/ 

INTACT : 

1 

TOL5/ 

INTACT : 

1 

TOL5/ 

INTACT : 

1 

TOL5/ 

INTACT : 

1 

TOL5/ 

INTACT ; 

1 

TOL5/ 

INTACT : 

2 

TOL5/ 

INTACT : 

2 

TOL5/ 

INTACT : 

2 

TOL5/ 

INTACT : 

2 

TOL5/ 

INTACT : 

2 

TOL5/ 

INTACT : 

2 , 

TOL5/ 

INTACT : 

2, 

TOL5/ 

INTACT : 

3 , 

TOL5/ 

INTACT : 

3 . 

TOL5/ 

INTACT ; 

3 . 

TOL5/ 

INTACT : 

3 . 

TOL5/ 

INTACT : 

3. 

TOL5/ 

INTACT : 

3. 

TOL5/ 

INTACT : 

3. 

TOL5/ 

INTACT : 

4. 

TOL5/ 

INTACT : 

4. 

TOL5/ 

INTACT : 

4. 

BIG  CTY5 

1. 

NO  SCIEN 

BIG  CTY5 

2 . 

LIFESCI 

BIG  CTY5 

3 . 

PHYSSCI 

BIG  CTY5 

4 . 

EATHSCI 

BIG  CTY5 

5. 

GEN  SCI 

BIG  CTY5 

6. 

INTESCI 

BIG  CTY5 

7 . 

MISSING 

MID  CTY5 

1. 

NO  SCIEN 

MID  CTY5 

2 . 

LIFESCI 

MID  CTY5 

3 . 

PHYSSCI 

MID  CTY5 

4. 

EATHSCI 

MID  CTY5 

5. 

GEN  SCI 

MID  CTY5 

6. 

INTESCI 

MID  CTY5 

7. 

MISSING 

FR/BTWN5 

1. 

NO  SCIEN 

FR/BTWN5 

2. 

LIFESCI 

FR/BTWN5 

3 . 

PHYSSCI 

FR/BTWN5 

4. 

EATHSCI 

FR/BTWN5 

5. 

GEN  SCI 

FR/BTWN5 

6. 

INTESCI 

FR/BTWN5 

7. 

MISSING 

SML  TWN5 

1. 

NO  SCIEN 

SML  TWN5 

2. 

LIFESCI 

SML  TWN5 

3. 

PHYSSCI 

Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


025 

T/ 

44 

(44 

026 

T/ 

45 

(45‘ 

027 

T/ 

46 

(46 

02S 

T/ 

47 

(47 

029 

T/ 

51 

(51 

030 

T/ 

52 

(52 

031 

T/ 

53 

(53 

032 

T/ 

54 

(54 

033 

T/ 

55 

(55 

034 

T/ 

56 

(56 

035 

T/ 

57 

(57 

222222222222223222222222 

222222222222222322222222 

222222222222222232222222 

222222222222222223222222 

222222222222222222111111 

222222222222222222322222 

222222222222222222232222 

222222222222222222223222 

222222222222222222222322 

222222222222222222222232 

222222222222222222222223 


TOL5/ 

INTACT : 

4. 

SML  TWN5 

4. 

EATHSCI 

TOL5/ 

INTACT : 

4. 

SML  TWN5 

5. 

GEN  SCI 

TOL5/ 

INTACT : 

4. 

SML  TWN5 

6. 

INTESCI 

TOL5/ 

INTACT : 

4. 

SML  TWN5 

7. 

MISSING 

TOL5/ 

INTACT : 

5. 

RURAL 5 

1. 

NO  SCIEN 

TOL5/ 

INTACT : 

5. 

RURAL5 

2. 

LIFESCI 

TOL5/ 

INTACT : 

5. 

RURAL5 

3. 

PHYSSCI 

TOL5/ 

INTACT: 

5. 

RURAL5 

4. 

EATHSCI 

TOL5/ 

INTACT : 

5. 

RURAL 5 

5. 

GEN  SCI 

TOL5/ 

INTACT : 

5. 

RURAL5 

6. 

INTESCI 

TOL5/ 

INTACT: 

5. 

RURAL5 

7. 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0068 
INTERACTION: 
NOS,  SOS 
SCHT/ 

N/A 

INTERACTION 


001  S/ 

11 

(11 

) -l-l-l-l-l-l-l-l-l-l-l-l 

002  S/ 

12 

(12 

) -10000000000-10000000000 

003  S/ 

13 

(13 

) 00-10000000000-100000000 

004  S/ 

14 

(14 

) 0000-10000000000-1000000 

005  S/ 

15 

(15 

) 000000-10000000000-10000 

006  S/ 

16 

(16 

) 00000000-10000000000-100 

007  S/ 

17 

(17 

) 0000000000-10000000000-1 

OOS  S/ 

21 

(21 

) -1-1-1-1-1-1000000000000 

009  S/ 

22 

(22 

) 010000000000000000000000 

010  S/ 

23 

(23 

) 000100000000000000000000 

Oil  S/ 

24 

(24 

) 000001000000000000000000 

012  S/ 

25 

(25 

) 000000010000000000000000 

013  S/ 

26 

(26 

) 000000000100000000000000 

014  S/ 

27 

(27 

) 000000000001000000000000 

015  S/ 

31 

(31 

) 000000000000010101010101 

016  S/ 

32 

(32 

) 000000000000010000000000 

017  S/ 

33 

(33 

) 000000000000000100000000 

OIS  S/ 

34 

(34 

) 000000000000000001000000 

019  S/ 

35 

(35 

) 000000000000000000010000 

020  S/ 

36 

(36 

) 000000000000000000000100 

021  S/ 

37 

(37 

) 000000000000000000000001 

CONDITIONING 

VARIABLE  ID 

: BACK0061 

DESCRIPTION: 
GRADES/ASSESSMENTS : 

INTERACTION:  GENDER  BY  NUMBE 

N12 

CONDITIONING 

VAR  LABEL: 

GEND/SEM 

NAEP  ID: 
TYPE  OF 

CONTRAST : 

N/A 

INTERACTION 

001  G/S 

11 

(11 

) 01010101 

002  G/S 

12 

(12 

) -1000000 

003  G/S 

13 

(13 

) 00-10000 

004  G/S 

14 

(14 

) 0000-100 

005  G/S 

15 

(15 

) 0000-100 

006  G/S 

21 

(21 

) -l-l-l-l 

007  G/S 

22 

(22 

) 01000000 

OOS  G/S 

23 

(23 

) 00010000 

009  G/S 

24 

(24 

) 00000100 

010  G/S 

25 

(25 

) 00000100 

CONDITIONING 

VARIABLE  IE 

1:  BACK0063 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

INTERACTION:  RACE /ETHNICITY 

N12 

CONDITIONING 

VAR  LABEL: 

RACE/SEM 

NAEP  ID; 
TYPE  OF 

CONTRAST : 

N/A 

INTERACTION 

001  R/S 

11 

(11 

) 010101010101010101010101 

002  R/S 

12 

(12 

) -1000000-1000000-1000000 

003  R/S 

13 

(13 

) 00-1000000-1000000-10000 

004  R/S 

14 

(14 

) 0000-1000000-1000000-100 

005  R/S 

15 

(15 

) 000000-1000000-1000000-1 

006  R/S 

21 

(21 

) -1-1-1000000000000000000 

007  R/S 

22 

(22 

) 010000000000000000000000 

OOS  R/S 

23 

(23 

) 000100000000000000000000 

009  R/S 

24 

(24 

) 000001000000000000000000 

010  R/S 

25 

(25 

) 000000010000000000000000 

Oil  R/S 

31 

(31 

) 000000-1-1-1000000000000 

012  R/S 

32 

(32 

) 000000000100000000000000 

013  R/S 

33 

(33 

) 000000000001000000000000 

014  R/S 

34 

(34 

) 000000000000010000000000 

015  R/S 

35 

(35 

) 000000000000000100000000 

016  R/S 

41 

(41 

) 000000000000-1-1-1000000 

017  R/S 

42 

(42 

) 000000000000000001000000 

OIS  R/S 

43 

(43 

) 000000000000000000010000 

019  R/S 

44 

(44 

) 000000000000000000000100 

020  R/S 

45 

(45 

) 000000000000000000000001 

SCHOOL  TYPE  BY  SCIENCE  COURSES  TAKING  THIS  YEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  21 

NUMBER  OF  INDEPENDENT  CONTRASTS:  12 


SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 

SCHT/ 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  10 

NUMBER  OF  INDEPENDENT  CONTRASTS:  4 


INTACT : 

1. 

PUBLIC 

1. 

NO  SCIEN 

INTACT : 

1. 

PUBLIC 

2. 

LIFESCI 

INTACT : 

1. 

PUBLIC 

3. 

PHYSSCI 

INTACT : 

1. 

PUBLIC 

4. 

EATHSCI 

INTACT : 

1. 

PUBLIC 

5. 

GEN  SCI 

INTACT : 

1. 

PUBLIC 

6. 

INTESCI 

INTACT : 

1. 

PUBLIC 

7. 

MISSING 

INTACT : 

2. 

PRIVATE 

1. 

NO  SCIEN 

INTACT : 

2 . 

PRIVATE 

2. 

LIFESCI 

INTACT : 

2 . 

PRIVATE 

3. 

PHYSSCI 

INTACT : 

2. 

PRIVATE 

4. 

EATHSCI 

INTACT : 

2. 

PRIVATE 

5. 

GEN  SCI 

INTACT : 

2. 

PRIVATE 

6. 

INTESCI 

INTACT : 

2. 

PRIVATE 

7. 

MISSING 

INTACT : 

3. 

CATHOLIC 

1. 

NO  SCIEN 

INTACT : 

3. 

CATHOLIC 

2. 

LIFESCI 

INTACT : 

3. 

CATHOLIC 

3. 

PHYSSCI 

INTACT : 

3. 

CATHOLIC 

4. 

EATHSCI 

INTACT : 

3. 

CATHOLIC 

5. 

GEN  SCI 

INTACT : 

3. 

CATHOLIC 

6. 

INTESCI 

INTACT : 

3. 

CATHOLIC 

7. 

MISSING 

GEND/SEM 

INTACT : 

1. 

MALE 

1.  SEMSCI-? 

GEND/SEM 

INTACT : 

1. 

MALE 

2.  SEMSCI-'' 

GEND/SEM 

INTACT : 

1. 

MALE 

3.  SEMSCI-'' 

GEND/SEM 

INTACT : 

1. 

MALE 

4.  SEMSCI-'' 

GEND/SEM 

INTACT : 

1. 

MALE 

5.  SEMSCI-'' 

GEND/SEM 

INTACT : 

2. 

FEMALE 

1.  SEMSCI-? 

GEND/SEM 

INTACT : 

2. 

FEMALE 

2.  SEMSCI-'' 

GEND/SEM 

INTACT : 

2. 

FEMALE 

3.  SEMSCI-'' 

GEND/SEM 

INTACT : 

2. 

FEMALE 

4.  SEMSCI-'' 

GEND/SEM 

INTACT : 

2. 

FEMALE 

5.  SEMSCI-'' 

NUMBER  OF 

SEMESTERS 

SCIENCE 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


20 

12 


RACE/SEM 

INTACT: 

1. 

WHI/Al/0 

1. 

SEMSCI-? 

RACE/SEM 

INTACT : 

1. 

WHI/AI/0 

2 . 

SEMSCI-^ 

RACE/SEM 

INTACT : 

1. 

WHI/AI/0 

3. 

SEMSCI-^ 

RACE/SEM 

INTACT : 

1. 

WHI/AI/0 

4. 

SEMSCI-'' 

RACE/SEM 

INTACT : 

1. 

WHI/AI/0 

5. 

SEMSCI-^ 

RACE/SEM 

INTACT : 

2. 

BLACK 

1. 

SEMSCI-? 

RACE/SEM 

INTACT : 

2. 

BLACK 

2. 

SEMSCI-^ 

RACE/SEM 

INTACT : 

2. 

BLACK 

3. 

SEMSCI-^' 

RACE/SEM 

INTACT : 

2. 

BLACK 

4. 

SEMSCI-'' 

RACE/SEM 

INTACT : 

2. 

BLACK 

5. 

SEMSCI-'' 

RACE/SEM 

INTACT : 

3. 

HISPANIC 

1. 

SEMSCI-? 

RACE/SEM 

INTACT : 

3. 

HISPANIC 

2 . 

SEMSCI-'' 

RACE/SEM 

INTACT: 

3. 

HISPANIC 

3. 

SEMSCI-'' 

RACE/SEM 

INTACT : 

3. 

HISPANIC 

4. 

SEMSCI-'' 

RACE/SEM 

INTACT : 

3. 

HISPANIC 

5. 

SEMSCI-'' 

RACE/SEM 

INTACT: 

4. 

ASIAN 

1. 

SEMSCI-? 

RACE/SEM 

INTACT : 

4. 

ASIAN 

2 . 

SEMSCI-'' 

RACE/SEM 

INTACT : 

4. 

ASIAN 

3. 

SEMSCI-'' 

RACE/SEM 

INTACT: 

4. 

ASIAN 

4. 

SEMSCI-'' 

RACE/SEM 

INTACT : 

4. 

ASIAN 

5. 

SEMSCI-'' 

ERIC 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0065 

INTERACTION: 

N12 

PARE/SEM 

N/A 

INTERACTION 


PARENTS'  EDUCATION  BY  NUMBER  OF  SEMESTERS  SCIENCE 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


25 

16 


001 

P/S 

11 

(11 

002 

P/S 

12 

(12 

003 

P/S 

13 

(13 

004 

P/S 

14 

(14 

005 

P/S 

15 

(15 

006 

P/S 

21 

(21 

007 

P/S 

22 

(22 

008 

P/S 

23 

(23 

009 

P/S 

24 

(24 

010 

P/S 

25 

(25 

Oil 

P/S 

31 

(31 

012 

P/S 

32 

(32 

013 

P/S 

33 

(33 

014 

P/S 

34 

(34 

015 

P/S 

35 

(35 

016 

P/S 

41 

(41 

017 

P/S 

42 

(42 

018 

P/S 

43 

(43 

019 

P/S 

44 

(44 

020 

P/S 

45 

(45 

021 

P/S 

51 

(51 

022 

P/S 

52 

(52 

023 

P/S 

53 

(53 

024 

P/S 

54 

(54 

025 

p/s 

55 

(55 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 


01010101010101010101010101010101  PARE/SEM  INTACT : 
-1000000-1000000-1000000-1000000  PARE/SEM  INTACT : 
00-1000000-1000000-1000000-10000  PARE/SEM  INTACT: 
0000-1000000-1000000-1000000-100  PARE/SEM  INTACT : 
000000-1000000-1000000-1000000-1  PARE/SEM  INTACT: 
“1“1“100000000000000000000000000  PARE/SEM  INTACT: 
01000000000000000000000000000000  PARE/SEM  INTACT : 
00010000000000000000000000000000  PARE/SEM  INTACT : 
00000100000000000000000000000000  PARE/SEM  INTACT : 
00000001000000000000000000000000  PARE/SEM  INTACT : 
000000-1-1-100000000000000000000  PARE/SEM  INTACT : 
00000000010000000000000000000000  PARE/SEM  INTACT : 
00000000000100000000000000000000  PARE/SEM  INTACT : 
00000000000001000000000000000000  PARE/SEM  INTACT : 
00000000000000010000000000000000  PARE/SEM  INTACT : 
000000000000-1-1-100000000000000  PARE/SEM  INTACT: 
00000000000000000100000000000000  PARE/SEM  INTACT : 
00000000000000000001000000000000  PARE/SEM  INTACT: 
00000000000000000000010000000000  PARE/SEM  INTACT: 
00000000000000000000000100000000  PARE/SEM  INTACT : 
000000000000000000-1-1-100000000  PARE/SEM  INTACT : 
00000000000000000000000001000000  PARE/SEM  INTACT : 
00000000000000000000000000010000  PARE/SEM  INTACT : 
OOOOOOOOOOOOOOOOOOOOOOOOOOOQOlOO  PARE/SEM  INTACT : 
00000000000000000000000000000001  PARE/SEM  INTACT : 


1.  < HS 
1.  < HS 
1.  < HS 
1.  < HS 

1.  < HS 

2.  HS  GRAD 
2.  HS  GRAD 
2.  HS  GRAD 
2.  HS  GRAD 

2.  HS  GRAD 

3 . POST  HS 
3 . POST  HS 
3 . POST  HS 
3 . POST  HS 

3 . POST  HS 

4,  COL  GRAD 
4,  COL  GRAD 
4.  COL  GRAD 
4 . COL  GRAD 

4.  COL  GRAD 

5.  PARED-? 

5 . PARED-? 

5 . PARED- ? 
5.  PARED-? 
5.  PARED-? 


1.  SEMSCI-? 

2.  SEMSCI-'" 

3.  SEMSCI-'" 

4.  SEMSCI-'" 

5.  SEMSCI-'" 

1.  SEMSCI-? 

2.  SEMSCI-'' 

3.  SEMSCI-'' 

4.  SEMSCI-'' 

5.  SEMSCI-'' 

1.  SEMSCI-? 

2.  SEMSCI-'' 

3.  SEMSCI-'' 

4.  SEMSCI-'' 

5.  SEMSCI-'' 

1.  SEMSCI-? 

2.  SEMSCI-'' 

3.  SEMSCI-'' 

4.  SEMSCI-'' 

5.  SEMSCI-'' 

1.  SEMSCI-? 

2.  SEMSCI-'' 

3.  SEMSCI-'' 

4.  SEMSCI-'' 

5.  SEMSCI-'' 


CONDITIONING  VARIABLE  ID:  BACK0067 

DESCRIPTION:  INTERACTION: 

GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL:  TOL5/SEM 

NAEP  ID:  N/A 

TYPE  OF  CONTRAST:  INTERACTION 


TYPE  OF  LOCALE  (5  CATEGORIES)  BY  NUMBER  OF  SEMESTERS  SCIENCE 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  25 

NUMBER  OF  INDEPENDENT  CONTRASTS:  16 


001 

T/S 

11 

(11 

002 

T/S 

12 

(12 

003 

T/S 

13 

(13 

004 

T/S 

14 

(14 

005 

T/S 

15 

(15 

006 

T/S 

21 

(21 

007 

T/S 

22 

(22 

008 

T/S 

23 

(23 

009 

T/S 

24 

(24 

010 

T/S 

25 

(25 

Oil 

T/S 

31 

(31 

012 

T/S 

32 

(32 

013 

T/S 

33 

(33 

014 

T/S 

34 

(34 

015 

T/S 

35 

(35 

016 

T/S 

41 

(41 

017 

T/S 

42 

(42 

018 

T/S 

43 

(43 

019 

T/S 

44 

(44 

020 

T/S 

45 

(45 

021 

T/S 

51 

(51 

022 

T/S 

52 

(52 

023 

T/S 

53 

(53 

024 

T/S 

54 

(54 

025 

T/S 

55 

(55 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 

) 


01010101010101010101010101010101  TOL5/SEM 
-1000000-1000000-1000000-1000000  TOL5/SEM 
00-1000000-1000000-1000000-10000  TOL5/SEM 
0000-1000000-1000000-1000000-100  TOL5/SEM 
000000-1000000-1000000-1000000-1  TOL5/SEM 
-1-1-100000000000000000000000000  TOL5/SEM 
01000000000000000000000000000000  TOL5/SEM 
00010000000000000000000000000000  TOL5/SEM 
00000100000000000000000000000000  TOL5/SEM 
00000001000000000000000000000000  TOL5/SEM 
000000-1-1-100000000000000000000  TOL5/SEM 
00000000010000000000000000000000  TOL5/SEM 
00000000000100000000000000000000  TOL5/SEM 
00000000000001000000000000000000  TOL5/SEM 
00000000000000010000000000000000  TOL5/SEM 
000000000000-1-1-100000000000000  TOL5/SEM 
00000000000000000100000000000000  TOL5/SEM 
00000000000000000001000000000000  TOL5/SEM 
00000000000000000000010000000000  TOL5/SEM 
00000000000000000000000100000000  TOL5/SEM 
000000000000000000-1-1-100000000  TOL5/SEM 
00000000000000000000000001000000  TOL5/SEM 
00000000000000000000000000010000  TOL5/SEM 
00000000000000000000000000000100  TOL5/SEM 
00000000000000000000000000000001  TOL5/SEM 


INTACT : 1 . 
INTACT : 1 . 
INTACT : 1 . 
INTACT : 1 . 
INTACT : 1 : 
INTACT:  2, 
INTACT:  2, 
INTACT:  2, 
INTACT:  2. 
INTACT:  2, 
INTACT:  3. 
INTACT:  3. 
INTACT:  3. 
INTACT:  3. 
INTACT:  3. 
INTACT : 4 . 
INTACT:  4. 
INTACT:  4. 
INTACT : 4 . 
INTACT:  4. 
INTACT : 5 . 
INTACT:  5. 
INTACT:  5. 
INTACT : 5 . 
INTACT : 5 . 


BIG  CTY5  1. 
BIG  CTY5  2, 
BIG  CTY5  3 , 
BIG  CTY5  4. 
BIG  CTY5  5, 
MID  CTY5  1. 
MID  CTY5  2. 
MID  CTY5  3, 
MID  CTY5  4, 
MID  CTY5  5. 
FR/BTWN5  1. 
FR/BTWN5  2. 
FR/BTWN5  3. 
FR/BTWN5  4. 
FR/BTWN5  5. 
SML  TWN5  1. 
SML  TWN5  2. 
SML  TWN5  3. 
SML  TWN5  4. 
SML  TWN5  5. 
RURAL5  1 . 
RURAL5  2 . 
RURAL5  3 . ■ 
RURAL5  4 . 
RURAL5  5 . 


SEMSCI-? 

SEMSCI-'' 

SEMSCI-'^ 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-? 

SEMSCI-'^ 

SEMSCI-'^ 

SEMSCI-'^ 

SEMSCI-'^ 

SEMSCI-? 

SEMSCI-'^ 

SEMSCI-'^ 

SEMSCI-'^ 

SEMSCI-'^ 

SEMSCI-? 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-'^ 

SEMSCI-? 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-'' 


CONDITIONING  VARIABLE  ID:  BACK0069 

DESCRIPTION:  INTERACTION: 

GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL:  SCHT/SEM 

NAEP  ID:  N/A 

TYPE  OF  CONTRAST:  INTERACTION 


SCHOOL  TYPE  BY  NUMBER  OF  SEMESTERS  SCIENCE 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


15 

8 


001 

S/S 

11 

(11 

002 

S/S 

12 

(12 

003 

S/S 

13 

(13 

004 

S/S 

14 

(14 

005 

S/S 

15 

(15 

006 

S/S 

21 

(21 

007 

S/S 

22 

(22 

008 

S/S 

23 

(23 

009 

S/S 

24 

(24 

010 

S/S 

25 

(25 

Oil 

s/s 

31 

(31 

012 

s/s 

32 

(32 

013 

s/s 

33 

(33 

014 

s/s 

34 

(34 

) 0101010101010101 
) -1000000-1000000 
) 00-1000000-10000 
) 0000-1000000-100 
) 000000-1000000-1 
) -1-1-10000000000 
) 0100000000000000 
) 0001000000000000 
) 0000010000000000 
) 0000000100000000 
) 000000-1-1-10000 
) 0000000001000000 
) 0000000000010000 
) 0000000000000100 


SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 

SCHT/SEM 


INTACT : 1 . 
INTACT : 1 . 
INTACT : 1 . 
INTACT : 1 . 
INTACT : 1 . 
INTACT:  2. 
INTACT:  2. 
INTACT:  2. 
INTACT : 2 . 
INTACT:  2. 
INTACT:  3. 
INTACT:  3. 
INTACT:  3. 
INTACT:  3. 


PUBLIC  1 . 
PUBLIC  2 . 
PUBLIC  3 . 
PUBLIC  4 . 
PUBLIC  5 . 
PRIVATE  1 . 
PRIVATE  2 . 
PRIVATE  3 . 
PRIVATE  4 . 
PRIVATE  5 . 
CATHOLIC  1. 
CATHOLIC  2. 
CATHOLIC  3 . 
CATHOLIC  4. 


SEMSCI-? 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-'' 

SEMSCI-? 

SEMSCI-'" 

SEMSCI-^ 

SEMSCI-'" 

SEMSCI-'^ 

SEMSCI-? 

SEMSCI-^ 

SEMSCI-'" 

SEMSCI-'^ 


(j'l 

ERIC 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


015  S/S  35 


(35 


) 0000000000000001 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SAMP  SI  (01 

002  SAMP  S2  (02 

003  SAMP  S3  (03 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


BACK0073 
SAMPLE  TYPE 
N04,  S04,  NOS, 

SUBSAMP 

CLASS 

) 00 
) 10 
) 01 


SOS,  N12 


SCHT/SEM  INTACT:  3.  CATHOLIC  5.  SEMSCI-'" 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SAMPLE  Si 
SAMPLE  S2  AND  S3 
SAMPLE  S3  AND  S3 


BACK0074 

INTERACTION:  SAMPLE  BY  RACE/ ETHNICITY 

N04,  S04,  NOS,  SOS,  N12 
/RAC 


NAEP 

TYPE 

ID: 

OF 

CONTRAST: 

N/A 

INTERACTION 

001 

/R 

11 

(11 

) 010101010101 

002 

/R 

12 

(12 

) -10000-10000 

003 

/R 

13 

(13 

) 00-10000-100 

004 

/R 

14 

(14 

) 0000-10000-1 

005 

/R 

21 

(21 

) -1-1-1000000 

006 

/R 

22 

(22 

) 010000000000 

007 

/R 

23 

(23 

) 000100000000 

OOB 

/R 

24 

(24 

) 000001000000 

009 

/R 

31 

(31 

) 000000-1-1-1 

010 

/R 

32 

(32 

) 000000010000 

Oil 

/R 

33 

(33 

) 000000000100 

012 

/R 

34 

(34 

) 000000000001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


12 

6 


/RAC 

INTACT : 

1. 

SAMP 

SI 

1. 

WHI/AI/0 

/RAC 

INTACT: 

1. 

SAMP 

SI 

2 . 

BLACK 

/RAC 

INTACT : 

1. 

SAMP 

Si 

3. 

HISPANIC 

/RAC 

INTACT : 

1. 

SAMP 

SI 

4. 

ASIAN 

/RAC 

INTACT : 

2. 

SAMP 

S2 

1. 

WHI/AI/0 

/RAC 

INTACT : 

2. 

SAMP 

S2 

2. 

BLACK 

/RAC 

INTACT : 

2 . 

SAMP 

S2 

3. 

HISPANIC 

/RAC 

INTACT : 

2 . 

SAMP 

S2 

4. 

ASIAN 

/RAC 

INTACT : 

3. 

SAMP 

S3 

1. 

WHI/AI/0 

/RAC 

INTACT: 

3. 

SAMP 

S3 

2 . 

BLACK 

/RAC 

INTACT : 

3. 

SAMP 

S3 

3. 

HISPANIC 

/RAC 

INTACT : 

3. 

SAMP 

S3 

4 . 

ASIAN 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  B00S901Y  (01 

002  B00S901N  (02 

003  B00S901C  (03 

004  B00S901M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0075 

DO  YOU  HAVE  YOUR  OWN  STUDY  DESK  OR  TABLE  AT  HOME? 
N04,  NOS,  SOS,  N12 


B00S901 

CLASS 

) 000 
) 100 
) 010 
) 001 

BACK0076 

HOW  SAFE  DO  YOU  FEEL  AT  SCHOOL? 
N04,  NOS,  SOS,  N12 

B009401 

CLASS 


001 

VRY  SAFE 

(01 

) 0000 

002 

SAFE 

(02 

) 1000 

003 

UNSAFE 

(03 

) 0100 

004 

VRUNSAFE 

(04 

) 0010 

005 

MISSING 

(M 

) 0001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

IDK 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

VERY  SAFE 
SOMEWHAT  SAFE 
SOMEWHAT  UNSAFE 
VERY  UNSAFE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  MOMHOM-Y  (1 

002  MOMHOM-N  (2 

003  MOMHOM-?  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES / ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0077 

DOES  MOTHER  OR  STEPMOTHER  LIVE  AT  HOME  WITH  YOU? 
N04,  NOB,  SOB,  N12 
MOM0HOME 


B005601 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


MOTHER  AT  HOME 
MOTHER  AT  HOME 
MOTHER  AT  HOME 


YES 

NO 

MISSING 


BACK007B 

DOES  FATHER  OR  STEPFATHER  LIVE  AT  HOME  WITH  YOU? 
N04,  NOB,  SOB,  •N12 
DAD0HOME 


001  DADHOM-Y  (1 

002  DADHOM-N  (2 

003  DADHOM-?  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


B005701 

CLASS 

) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


FATHER  AT  HOME 
FATHER  AT  HOME 
FATHER  AT  HOME 


YES 

NO 

MISSING 


SUBJ0002 

AGREE /DISAGREE:  I LIKE  SCIENCE 
N04,  NOB,  SOB,  N12 

KBllOOl 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


o 

ERIC 


6^2 


BEST  COPY  AVAILABLE 


561 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  K811001A  (01 

) 000 

002  K811001B  (02 

) 100 

003  K811001C  (03 

) 010 

004  K811001M  (M 

) 001 

CONDITIONING  VARIABLE  ID: 

SUBJ0003 

DESCRIPTION: 

AGREE/DISAGREE 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  N08,  S08, 

NAEP  ID: 

K811002 

TYPE  OF  CONTRAST: 

CLASS 

001  K811002A  (01  ] 

1 000 

002  K811002B  (02  ) 

1 100 

003  K811002C  (03  ) 

1 010 

004  K811002M  (M  ) 

1 001 

AGREE 
NOT  SURE 
DISAGREE 
MISSING 


GOOD  AT  SCIENCE 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

AGREE 
NOT  SURE 
DISAGREE 
MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0004 

DESCRIPTION:  AGREE/DISAGREE:  LEARNING  SCI  MOSTLY  MEMORIZATION 

GRADES /ASSESSMENTS:  N04,  N08,  S08,  Nl2 


CONDITIONING  VAR  LABEL: 


NAEP  ID:  K811003 

TYPE  OF  CONTRAST:  CLASS 

001  K811003A  (01  ) 000 

002  K811003B  (02  ) 100 

003  K811003C  (03  ) 010 

004  K811003M  (M  ) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

AGREE 
NOT  SURE 
DISAGREE 
MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0005 

DESCRIPTION:  AGREE/DISAGREE:  SCI  USEFUL  FOR  EVERYDAY  PROBLEMS 

GRADES /ASSESSMENTS:  N04,  N08,  S08,  Nl2 


CONDITIONING  VAR  LABEL: 


NAEP  ID:  K811004 

TYPE  OF  CONTRAST:  CLASS 

001  K811004A  (01  ) 000 

002  K811004B  (02  ) 100 

003  K811004C  (03  ) 010 

004  K811004M  (M  ) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

AGREE 
NOT  SURE 
DISAGREE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0006 

AGREE/ DISAGREE:  IF  CHOICE,  WOULD  NOT  STUDY  SCIENCE 
N04,  N08,  £08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


K811005 

CLASS 


001  K811005A  (01  ) 000 

002  K811005B  (02  ) 100 

003  K811005C  (03  ) 010 

004  K811005M  (M  ) 001 


AGREE 
NOT  SURE 
DISAGREE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0007 

AGREE/ DISAGREE:  ALL  CAN  DO  WELL  IN  SCI  IF  THEY  TRY 
N04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


K811006 

CLASS 


001  K811006A  (01  ) 000 

002  K811006B  (02  ) 100 

003  K811006C  (03  ) 010 

004  K811006M  (M  ) 001 


AGREE 
NOT  SURE 
DI SAGREE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0008 

AGREE/DISAGREE:  SCIENCE  IS  BORING 
N04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


K811007 

CLASS 


001  K811007A  (01  ) 000 

002  K811007B  (02  ) 100 

003  K811007C  (03  ) 010 

004  K811007M  (M  ) 001 


AGREE 
NOT  SURE 
DISAGREE 
MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0009 

DESCRIPTION:  AGREE/DISAGREE:  SCIENCE  IS  A HARD  SUBJECT 

GRADES/ASSESSMENTS : N04,  N08 , S08,  Nl2 

CONDITIONING  VAR  LABEL: 

K811008 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


NAEP  ID: 

TYPE  OF  CONTRAST: 


001 

K811008A 

(01 

) 000 

002 

K811008B 

(02 

) 100 

003 

K811008C 

(03 

) 010 

AGREE 
NOT  SURE 
DISAGREE 


0"2 

ERIC 


4 

3 


4 

3 


4 

3 


4 

3 


4 

3 


4 

3 


4 

3 


T O 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


004  K811008M  (M  ) 

001  MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJOOlO 

EVER  DONE  HANDS-ON  PROJECT  WITH  LIVING  THINGS? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811101  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811101Y  (01  ) 

002  K811101M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJOOll 

EVER  DONE  HANDS-ON  PROJECT  WITH  ELECTRICITY? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K8 11102  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811102Y  (01  ) 

002  K811102M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0012 

EVER  DONE  HANDS-ON  PROJECT  WITH  CHEMICALS? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811103  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811103Y  (01  ) 

002  K811103M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0013 

EVER  DONE  HANDS-ON  PROJECT  WITH  ROCKS  OR  MINERALS? 
N04,  N08,  S08 ■ 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811104  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811104Y  (01  ) 

002  K811104M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0014 

DONE  HANDS-ON  PROJ  W/  MAGNIFYING  GLASS/MICROSCOPE? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811105  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811105Y  (01  1 

002  K811105M  (M  1 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0015 

DONE  HANDS-ON  PROJ  W/  THERMOMETER  OR  BAROMETER? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811106  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811106Y  (01  1 

002  K811106M  (M 

1 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0016 

EVER  DONE  HANDS-ON  PROJECT  WITH  SIMPLE  MACHINES? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811107  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811107Y  (01 

002  K811107M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0017 

HAVE  DONE  HANDS-ON  PROJECT  WITH  NONE  OF  THE  ABOVE? 
N04,  N08,  S08 

NAEP  ID: 

TYPE  OF  CONTRAST: 

K811108  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  K811108Y  (01 

002  K811108M  (M 

) 0 yes 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SUBJ0018 

HOW  OFTEN  DO  YOU  STUDY  SCIENCE  IN  SCHOOL? 
N04,  N08,  S08 

NAEP  ID: 

K811201  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  6 

614 


BEST  copy  AVAILABLE 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 


001  K811201A  (01  ) 00000 

002  K811201B  (02  ) lOOOO 

003  K811201C  (03  ) 01000 

004  K811201D  (04  ) 00100 

005  K811201E  (05  ) 00010 

006  K811201M  (M  ) 00001 


NUMBER  OF  INDEPENDENT  CONTRASTS 

EVERY  DAY 

3-4  times  a week 

1-2  TIMES  A WEEK 

LESS  THEN  ONCE  WEEK 

NEVER 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0019 

HOW  MUCH  TIME  PER  WEEK  DOING  SCIENCE  HOMEWORK’ 
N04 


K811301 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K811301A  (01 

002  K811301B  (02 

003  K811301C  (03 

004  K811301D  (04 

005  K811301E  (05 

006  K811301F  (06 

007  K811301M  (M 


) 000000 
) 100000 
) 010000 
) 001000 
) 000100 
) 000010 
) 000001 


DON'T  HAVE  SCIENCE 

NONE 

1/2  HOUR 

1 HOUR 

2 HOURS 

MORE  THAN  2 HOURS 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0020 

DO  SCI  PROJECTS  IN  SCHOOL  THAT  TAKE  1 OR  MORE  WKS’ 
N04,  N08,  S08 


K811401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  K811401Y  (01  ) 00 

002  K811401N  (02  ) 10 

003  K811401M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0021 

LAST  2 YRS,  BEEN  IN  SCI  FAIR,  FESTIVAL,  SCI  DAY’ 
N04,  N08,  S08,  N12 


K811501 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  K811501Y  (01  ) 00 

002  K811501N  (02  ) 10 

003  K811501M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0022 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DO  YOU  READ  TEXTBOOK’ 
N04,  N08,  S08,  N12 


K811601 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  K811601A  (01  ) 0000 

002  K811601B  (02  ) 1000 

003  K811601C  (03  ) OlOO 

004  K811601D  (04  ) 0010 

005  K811601M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SUBJ0023 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DO  YOU  READ  MAGS/BKS? 
N04,  N08,  S08,  N12 


K811602 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K811602A  (01  ) 0000 

002  K811602B  (02  ) 1000 

003  K811602C  (03  ) 0100 

004  K811602D  (04  ) 0010 

005  K811602M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST: 


SUBJ0024 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DISCUSS  SCIENCE  NEWS’ 
N04,  N08,  S08,  N12 


K811603 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  K811603A  (01  ) 0000 

002  K811603B  (02  ) 1000 

003  K811603C  (03  ) 0100 

004  K811603D  (04  ) 0010 

005  K811603M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 


SUBJ0025 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  WORK  WITH  OTHERS? 


5 


7 

6 


3 

2 


3 

2 


5 

4 


5 

4 


ERIC 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  K811604A 

002  K811604B 

003  K811604C 

004  K811604D 

005  K811604M 


(01 

(02 

(03 

(04 

(M 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


N04,  N08,  S08,  N12 

K811604 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 

SUBJ0026 


001  K811605A  (01 

002  K811605B  (02 

003  K811605C  (03 


004  K811605D  (04 

005  K811605M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  K811606A  (01 

002  K811606B  (02 

003  K811606C 


(03 

004  K811606D  (04 

005  K811606M  (M 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


FOR  SCI  IN  SCHOOL,  HOW  OFTEN  GIVE  ORAL  REPORT? 
N04,  N08,  S08,  N12 


K811605 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


SUBJ0027 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  GIVE  WRITTEN  REPORT? 
N04,  N08,  S08,  N12 


K811606 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0028 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  K811607A  (01 

002  K811607B  (02 

003  K811607C  (03 

004  K811607D  (04 

005  K811607M  (M 

CONDITIONING  VARIABLE  IE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  K811608A  (01 

002  K811608B  (02 

003  K811608C  (03 

004  K811608D  (04 

005  K811608M  (M 

CONDITIONING  VARIABLE  IE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  K811609A  (01 

002  K811609B  (02 

003  K811609C  (03 

004  K811609D  (04 

005  K811609M  (M 


FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DO  HANDS-ON  PROJECT? 
N04 


K811607 

CLASS 

} 0000 


> 1000 
) 0100 
) 0010 
) 0001 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


SUBJ0029 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DISCUSS  RESULTS? 
N04 


K811608 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


SUBJ0030 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DO  YOU  USE  COMPUTER? 
N04,  N08,  S08,  N12 


K811609 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


SUBJ0031 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  TAKE  TEST  OR  QUIZ? 
N04,  N08,  S08,  N12 


K811610 


TYPE  OF  CONTRAST: 

CLASS 

001 

K811610A 

(01 

) 0000 

002 

K811610B 

(02 

) 1000 

003 

K811610C 

(03 

) 0100 

004 

K811610D 

(04 

) 0010 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 


ERIC 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


005  K811610M  (M  ) 0001 


MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0032 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  DO  YOU  USE  LIBRARY? 
N04,  NOS,  SOS,  N12 


KS11611 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  KS11611A  (01  ) 0000 

002  KS11611B  (02  ) 1000 

003  KS11611C  (03  ) 0100 

004  KS11611D  (04  ) 0010 

005  KS11611M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0033 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  OBSERVE /MEAS  OUTS  IDE 
N04,  NOS,  SOS,  N12 


KS11612 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  KS11612A  (01  ) 0000 

002  KS11612B  (02  ) 1000 

* 003  KS11612C  (03  ) 0100 

004  KS11612D  (04  ) 0010 

005  KS11612M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0034 

HOW  OFTEN  DOES  SCIENCE  TEACHER  TALK  TO  CLASS? 
N04,  NOS,  SOS,  N12 


KS11701 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  KS11701A  (01  ) 0000 

002  KS11701B  (02  ) 1000 

003  KS11701C  (03  ) 0100 

004  KS11701D  (04  ) 0010 

005  KS11701M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  KS11702A  (01  ) 0000 

002  KS11702B  (02  ) 1000 

003  KS11702C  (03  ) OlOO 

004  KS11702D  (04  ) 0010 

005  KS11702M  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


ID:  SUBJ0035 

HOW  OFTEN  DOES  SCIENCE  TEACHER  DO  DEMON  STRATI  ON  •> 
N04,  NOS,  SOS,  N12 

KS11702 
CLASS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0036 

HOW  OFTEN  DOES  SCIENCE  TEACHER  SHOW  VIDEO  OR  TV> 
N04,  NOS,  SOS,  N12 


KS11703 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  KS11703A  (01  ) 0000 

002  KS11703B  (02  ) 1000 

003  KS11703C  (03  ) 0100 

004  KS11703D  (04  ) 0010 

005  KS11703M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL:  • 

NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0037 

HOW  OFTEN  DOES  SCIENCE  TEACHER  USE  COMPUTER? 
N04,  NOS,  SOS,  N12 


KS11704 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

KS11704A 

(01 

) 0000 

002 

KS11704B 

(02 

) 1000 

003 

KS11704C 

(03 

) 0100 

004 

KS11704D 

(04 

) 0010 

005 

KS11704M 

(M 

) 0001 

ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ003S 

HOW  OFTEN  DOES  SCI  TEACHER  USE  CD'S/LASER  DISCS'^ 
N04,  NOS,  SOS,  N12 


KS11705 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  K811705A  (01  ) 0000 

002  K811705B  (02  ) 1000 

003  K811705C  (03  ) 0100 

004  K811705D  (04  ) 0010 

005  K811705M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0039 

HOW  OFTEN  DOES  SCI  CLASS  GO  ON  A FIELD  TRIP? 
N04,  N08,  S08,  N12 


K811801 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K811801A  (01  ) 000 

002  K811801B  (02  ) 100 

003  K811801C  (03  ) 010 

004  K811801M  (M  ) 001 


3 OR  MORE  A DAY 
1 OR  2 TIMES  A YEAR 
NEVER  OR  HARDLY  EVER 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0040 

HOW  OFTEN  DOES  GUEST  SPEAKER  COME  TO  SCI  CLASS? 

N04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


K811901 

CLASS 


001  K811901A  (01  ) 000 

002  K811901B  (02  ) 100 

003  K811901C  (03  ) 010 

004  K811901M  (M  ) 001 


3 OR  MORE  A DAY 
1 OR  2 TIMES  A YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  SMOOIOIA  (01  ) 0000 

002  SMOOIOIB  (02  ) 1000 

003  SMOOIOIC  (03  ) 0100 

004  SMOOIOID  (04  . ) 0010 

005  SMOOIOIM  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ALMOST  ALL 
MORE  THAN  HALF 
ABOUT  HALF 
LESS  THAN  HALF 
MISSING 


SUBJ0041 

ABOUT  HOW  MANY  QUESTIONS  RIGHT  ON  TEST? 
N04,  N08,  S08,  N12 

SMOOlOl 
CLASS 


CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  SM00201A  (01 

002  SM00201B  (02 

003  SM00201C  (03 

004  SM00201D  (04 

005  SM00201M  (M 

CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  SM00301A  (01  ) 0000 

002  SM00301B  (02  ) 1000 

003  SM00301C  (03  ) 0100 

004  SM00301D  (04  ) 0010 

005  SM00301M  (M  ) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MUCH  HARDER 

HARDER  THAN  OTHERS 

ABOUT  AS  HARD 

NOT  AS  HARD  AS  OTHER 

missing 


ID:  SUBJ0042 

HOW  HARD  TEST  COMPARED  TO  THOSE  IN  SCHOOL? 

N04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MUCH  HARDER 
HARDER  THAN  OTHERS 
ABOUT  THE  SAME 
EASIER  THEN  OTHES 
MISSING 


HOW  HARD  DID  YOU  TRY  ON  TEST  COMPARED  TO  OTHERS? 
N04,  N08,  S08,  N12 

SM00301 
CLASS 


SM00201 
CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 

ID:  SUBJ0043 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0044 

HOW  IMPORTANT  WAS  IT  YOU  DO  WELL  ON  THIS  TEST? 
N04,  N08,  S08,  N12 


SM00401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  SM00401A  (01  ) 0000 

002  SM00401B  (02  ) 1000 

003  SM00401C  (03  ) 0100 

004  SM00401D  (04  ) 0010 

005  SM00401M  (M  ) 0001 


VERY  IMPORTANT 
IMPORTANT 

SOMEWHAT  IMPORTANT 
NOT  VERY  IMPORTANTER 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


SUBJ0045 

HOW  OFTEN  HAD  TO  WRITE  LONG  ANSWERS  TO  QSTS? 
N04,  N08,  S08.  N12 


SM00501 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


4 

3 


4 

3 


5 

4 


5 

4 


5 

4 


5 

4 


5 


618 


BEST  COPY  AVAILABLE 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 


CLASS 


001  SM00501A  (01 

) 0000 

002  SM00501B  (02 

) 1000 

003  SM00501C  (03 

) 0100 

004  SM00501D  (04 

) 0010 

005  SM00501M  (M 

) 0001 

CONDITIONING  VARIABLE  ID: 

BACK0079 

DESCRIPTION: 

DESCRIBE 

Grades /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

B009701 

TYPE  OF  CONTRAST: 

CLASS 

001  B009701A  (01 

) 000000 

002  B009701B  (02 

) 100000 

003  B009701C  (03 

) 010000 

004  B009701D  (04 

) 001000 

005  B009701E  (05 

) 000100 

006  B009701F  (06 

1 000010 

007  B009701M  (M 

1 000001 

CONDITIONING  VARIABLE  ID: 

BACKOOSO 

DESCRIPTION: 

HOW  FAR  11 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

B009S01 

TYPE  OF  CONTRAST: 

CLASS 

001  B009S01A  (01  ) 

000000 

002  B009S01B  (02  ) 

100000 

003  B009S01C  (03  ) 

010000 

004  B009S01D  (04  ) 

001000 

005  B009S01E  (05  ) 

000100 

006  B009S01F  (06  ) 

000010 

007  B009S01M  (M  ) 

000001 

CONDITIONING  VARIABLE  ID: 

BACK00S3 

DESCRIPTION: 

DOES  YOUR 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS.  SOS, 

NAEP  ID: 

B009601 

TYPE  OF  CONTRAST: 

CLASS 

001  B009601A  (01  ) 

0000 

002  B009601B  (02  ) 

1000 

003  B009601N  (03  ) 

0100 

004  B009601D  (04  ) 

0010 

005  B009601E  (M  ) 

0001 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

AT  LEAST  ONCE  A WEEK 
ONCE  OR  TWICE  A MNTH 
ONCE  OR  TWICE  A YEAR 
NEVERERY  IMPORTANTER 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

MOSTLY  A'S 
MOSTLY  B'S 
MOSTLY  C'S 
MOSTLY  D'S 
MOSTLY  BELOW  D'S  . 

CLASSES  NOT  GRADED 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  FINISH  HS 
GRADUATE  HS 
SOME  ED  PAST  HS 
GRADUATE  COLLEGE 
GO  GRADUATE  SCHOOL 
I DON'T  KNOW 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES,  FULL-TIME 
YES.  PART-TIME 
NO 

DON'T  LIVE  W/  MOTHER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0084 

DOES  YOUR  STEP/FATHER  WORK  AT  A JOB  FOR  PAY?  • 
NOS,  SOS,  N12 


B009602 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  B009602A  (01  ) 0000 

002  B009602B  (02  ) 1000 

003  B009602N  (03  ) 0100 

004  B009602D  (04  ) 0010 

005  B009602M  (M  ) 0001 


yes.  FULL-TIME 
yes.  PART-TIME 
NO 

DON'T  LIVE  W/  FATHER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING' VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0046 

DO  YOU/ TEACHER  SAVE  YOUR  SCI  WORK  IN  A PORTFOLIO’ 
NOS,  SOS 


KS12101 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  KS12101Y  (01  ) 00 

002  KS12101N  (02  ) 10 

003  KS12101M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING 

VARIABLE  ID: 

SUBJ0047 

DESCRIPTION: 

HOW  MUCH  TIME 

GRADES /ASSESSMENTS : 

NOS.  SOS.  N12 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

KS12201 

TYPE  OF  CONTRAST: 

CLASS 

001  KS12201A 

(01  ] 

1 0000000 

002  KS12201B 

(02  ) 

1 1000000 

003  KS12201C 

(03  ) 

1 OlO'OOOO 

004  KS12201D 

(04  ) 

1 0010000 

005  KS12201E 

(05  ) 

1 0001000 

WEEKLY  SPENT  ON  SCIENCE  HOMEWORK? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  TAKING  SCIENCE 

NONE 

1/2  HOUR 

1 HOUR 

2 HOURS 


4 


7 

6 


5 

4 


5 

4 


3 

2 


S 

7 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


006  K812201F  (06 

007  K812201G  (07 

008  K812201M  (M 


) 0000100 
) 0000010 
) 0000001 


3 HOURS 

MORE  THAN  3 HOURS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0048 

FOR  SCI  IN  SCHOOL,  HOW  OFTEN  HANDS-ON  ACTIVITIES? 
N08,  S08,  N12  ’ 


K811613 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K811613A  (01  ) 0000 

002  K811613B  (02  ) 1000 

003  K811613C  (03  ) 0100 

004  K811613D  (04  ) 0010 

005  K811613M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0049 

FOR  SCI,  HOW  OFTEN  DISCUSS  HANDS-ON  RESULTS? 
N08,  S08,  N12 


K811614 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K811614A  (01  ) 0000 

002  K811614B  (02  ) 1000 

003  K811614C  (03  ) 0100 

004  K811614D  (04  ) 0010 

005  K811614M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0050 

FOR  SCI,  DESIGN  & CARRY  OUT  OWN  INVESTIGATION? 

N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
class  number  of  INDEPENDENT  CONTRASTS: 


001  K811615A  (01  ) 0000 

002  K811615B  (02  ) 1000 

003  K811615C  (03  ) 0100 

004  K811615D  (04  ) 0010 

005  K811615M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID:  BACK0081 

DESCRIPTION:  DOES  YOUR  STEP/MOTHER  WORK  AT  A JOB  FOR  PAY? 


GRADES /ASSESSMENTS : N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  B009501 

TYPE  OF  CONTRAST:  CLASS 

001  B009501Y  (01  ) 000 

002  B009501N  (02  ) 100 

003  B009501C  (03  ) 010 

004  B009501M  (M  ) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

DON'T  LIVE  W/  MOTHER 
MISSING 


CONDITIONING  VARIABLE  ID:  BACK0082 

DESCRIPTION:  DOES  YOUR  STEP/ FATHER  WORK  AT  A JOB  FOR  PAY? 


GRADES /ASSESSMENTS:  N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  B009502 

TYPE  OF  CONTRAST:  CLASS 

001  B009502Y  (01  ) 000 

002  B009502N  (02  ) 100 

003  B009502C  (03  ) 010 

004  B009502M  (M  ) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

DON'T  LIVE  W/  FATHER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0085 

SINCE  KDG,  GRADES  ATTENDED  IN  THIS  STATE? 
N12 


B008301 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  B008301A  (01  ) 00000 

002  B008301B  (02  ) 10000 

003  B008301C  (03  ) 01000 

004  B008301D  (04  ) 00100 

005  B008301E  (05  ) 00010 

006  B008301M  (M  ) 00001 


LESS  THAN  1 GRADE 

1-2  GRADES 

3-5  GRADES 

6-9  GRADES 

MORE  THAN  9 GRADES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK0086 

DESCRIBE  YOUR  OVERALL  GRADES  SINCE  9TH  GRADE 
N12 


B009901 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


5 

4 


5 

4 


4 

3 


4 

3 


6 

5 


7 

6 


6.0 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  B009901A  (01 

) 000000 

002  B009901B  (02 

) 100000 

003  B009901C  (03 

) 010000 

004  B009901D  (04 

) 001000 

005  B009901E  (05 

) 000100 

006  B009901F  (06 

) 000010 

007  B009901M  (M 

) 000001 

CONDITIONING  VARIABLE  ID: 

BACK0087 

DESCRIPTION: 

WHICH  BEST  DESCRIBES  YOUR 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

B008501 

TYPE  OF  CONTRAST: 

CLASS 

001  B008501A  (01  ) 

1 0000 

002  B008501B  (02  ) 

1000 

003  B008501C  (03  ) 

0100 

004  B008501D  (04  ) 

0010 

005  B008501M  (M  ) 

0001 

CONDITIONING  VARIABLE  ID: 

BACK0088 

DESCRIPTION: 

WHAT  WILL  TAKE  LARGEST  AMT, 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

B005501 

TYPE  OF  CONTRAST: 

CLASS 

001  B005501A  (01  ) 

000000 

002  B005501B  (02  ) 

100000 

003  B005501C  (03  ) 

010000 

004  B005501D  (04  ) 

001000 

005  B005501E  (05  ) 

000100 

006  B005501F  (06  ) 

000010 

007  B005501M  (M  ) 

000001 

CONDITIONING  VARIABLE  ID: 

BACK0089 

DESCRIPTION: 

KIND  OF  WORK  DONE  BY  YOUR  M 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

MOTHOCC 

TYPE  OF  CONTRAST: 

CLASS 

MOSTLY  A'S 
MOSTLY  B'S 
MOSTLY  C'S 
MOSTLY  D'S 
MOSTLY  BELOW  D'S 
CLASSES  NOT  GRADED 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

GENERAL 

ACADEMIC/COLLEGE  PRE 
VOCATIONAL  OR  TECHN 
OTHER 
MISSING 


TIME  AFTER  HIGH-SCH 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

WORKING  FULL-TIME 
VOCA/TECH / BUS INESS 
ATTEND  2 YR  COLLEGE 
ATTEND  4YR  COLLEGE 
SERVING  IN  MILITARY 
OTHER 

missing 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001 

NOT  PRE 

(01, M 

002 

OFFICE  W 

(02 

003 

TECHNIC 

(03 

004 

PROT  SRV 

(04 

005 

SALES 

(05 

006 

OWNER 

(06 

007 

SERV  WRK 

(07 

008 

SKILL  T 

(08 

009 

LABORER 

(09 

010 

OPERATOR 

(10 

Oil 

FARMER 

(11 

012 

HOMEMAKE 

(12 

013 

MANAGER 

(13 

014 

MILITARY 

(14 

015 

TEACHER 

(15 

016 

PROFESSl 

(16 

017 

PROFESS2 

(17 

CONDITIONING  VARIABLE 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 0000000000000000 
.)  1000000000000000 
) 0100000000000000 
) 0010000000000000 
) oooiooopoooooooo 
) 0000100000000000 
) 0000010000000000 
) 0000001000000000 
) 0000000100000000 
) 0000000010000000 
) 0000000001000000 
) 0000000000100000 
) 0000000000010000 
) 0000000000001000 
) 0000000000000100 
) 0000000000000010 
) 0000000000000001 

: BACK0090 

KIND  OF  WORK  DONE 
N12 

BOllAOl 

CLASS 


NOT  PRESENT  IN  HOUSEHOLD 
OFFICE  WORKER 
TECHNICAL 

PROTECTIVE  SERVICE 

SALES 

OWNER 

SERVICE  WORKER 
SKILLED  TRADESPERSON 
LABORER 
OPERATOR 

FARMER/ FARM  MANAGER 

HOMEMAKER 

MANAGER 

MILITARY 

SCHOOL  TEACHER 

PROFESSIONAL  1 

PROFESSIONAL  2 


BY  YOUR  MOTHER/FEMALE  GUARDIAN? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  BOllAOlM  (M  ) 0 

002  BOllAOlN  (@  ) 1 


missing 

NOT  PRESENT  IN  HOUSEHOLD 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


BACK 00 91 

KIND  OF  WORK  DONE  BY  YOUR  FATHER /MALE  GUARDIAN’ 
N12 


FATHOCC 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  NOT  PRES  (01,M 

002  OFFICE  (02 

003  TECHNIC  (03 

004  PROT  SRV  (04 

005  SALES  (05 

006  OWNER  (06 

007  SERV  WRK  (07 

008  SKILL  TR  (08 


) 0000000000000000 
) 1000000000000000 
) 0100000000000000 
) 0010000000000000 
) 0001000000000000 
) 0000100000000000 
) 0000010000000000 
) ooooooioooooopoo 


NOT  PRESENT  IN  HOUSEHOLD 
OFFICE  WORKER 
TECHNICAL 

PROTECTIVE  SERVICES 

SALES 

OWNER 

SERVICE  WORKER 
SKILLED  TRADEPERSON 


ERIC 

“ ^ 6.1 


5 

4 


7 

6 


17 

16 


2 

1 


17 

16 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


009 

LABORER 

(09 

010 

OPERATOR 

(10 

Oil 

FARMER 

(11 

012 

HOMEMAKE 

(12 

013 

MANAGER 

(13 

014 

MILITARY 

(14 

015 

TEACHER 

(15 

016 

PROFESS 1 

(16 

017 

PROFESS 2 

(17 

CONDITIONING 

VARIABLE 

DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 0000000100000000 
) 0000000010000000 
) 0000000001000000 
) 0000000000100000 
) 0000000000010000 
) 0000000000001000 
) 0000000000000100 
) 0000000000000010 
) 0000000000000001 

BACK0092 

KIND  OF  WORK  DONE  BY 

N12 

B012A02 

CLASS 


LABORER 

OPERATOR 

FARMER/FARM  MANAGER 

HOMEMAKER 

MANAGER 

MILITARY 

SCHOOL  TEACHER 

PROFESSIONAL  1 

PROFESSIONAL  2 


YOUR  FATHER/MALE  GUARDIAN? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  B012A02M  (M  ) 0 

002  B012A02N  (@  ) 1 


missing 

NOT  PRESENT  IN  HOUSEHOLD 


CONDITIONING  VARIABLE  ID:  SUBJ0051 

DESCRIPTION:  DONE  SCHOOL  SCI  INVEST /PROJECTS  W/  LIVING  THINGS? 


GRADES /ASSESSMENTS:  Nl2 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  K812301 

TYPE  OF  CONTRAST:  CLASS 

001  K812301Y  (01  ) 0 

002*  K812301M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0052 

DESCRIPTION:  DONE  SCHOOL  SCI  INVEST/PROJECTS  W/  ELECTRICITY? 


GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  K812302 

TYPE  OF  CONTRAST:  CLASS 

001  K812302Y  (01  ) 0 

002  K812302M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0053 

DESCRIPTION:  DONE  SCHOOL  SCI  INVEST/ PROJECTS  W/  CHEMICALS? 


GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  K812303 

TYPE  OF  CONTRAST:  CLASS 

001  K812303Y  (01  ) 0 

002  K812303M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0054 

DESCRIPTION:  DONE  SCHOOL  SCI  INVEST/ PROJECTS  W/  ROCKS /MINERALS? 


GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  K812304 

TYPE  OF  CONTRAST:  CLASS 

001  K812304Y  (01  ) 0 

002  K812304M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0055 

DESCRIPTION:  DONE  SCI  INVEST /PROJECTS  W/  MAG.  GLASS/MICROSCOPE? 


GRADES /ASSESSMENTS:  N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  K812305 

TYPE  OF  CONTRAST:  CLASS 

001  K812305Y  (01  ) 0 

002  K812305M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0056 

DONE  SCI  INVEST/PROJECTS  W/  THERMOMETER/BAROMETER? 
N12 


K812306 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K812306Y  (01  ) 0 

002  K812306M  (M  ) 1 


YES 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0057 

DONE  SCI  INVEST/PROJECTS  W/  SIMPLE  MACHINES? 

N12 

1^812307  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

ChhSS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


BEST  COPY  AVAILABLE 


6.2 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  K812307Y  (01  ) 0 

002  K812307M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0058 

DONE  SCI  PROJECTS  W/  INSTRUMENTS  MEAS.  SPEED? 

N12 

K812308  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K812308Y  (01  ) 0 

002  K812308M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0059 

DONE  SCHOOL  SCI  INVEST/ PROJECTS  W/  NONE  OF  ABOVE? 

N12 

K812309  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K812309Y  (01  ) 0 

002  K812309M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0060 

ARE  YOU  TAKING  A SCIENCE  COURSE  THIS  YEAR'’ 

N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


K812401 

CLASS 


001  K812401Y  (01  ) 00 

002  K812401N  (02  ) 10 

003  K812401M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0061 

SINCE  9TH  GRADE.  HOW  MUCH  GENERAL  SCIENCE? 
N12 


K812501 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K812501A  (01  ) 0000 

002  K812501B  (02  ) 1000 

003  K812501C  (03  ) 0100 

004  K812501D  (04  ) 0010 

005  K812501M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0062 

SINCE  9TH  GRADE.  HOW  MUCH  EARTH  & SPACE  SCIENCE? 
N12 


K812502 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  K812502A  (01  ) 0000 

002  K812502B  (02  ) 1000 

003  K812502C  (03  ) 0100 

004  K812502D  (04  ) 0010 

005  K812502M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


001  K812503A  (01  ) 0000 

002  K812503B  (02  ) 1000 

003  K812503C  (03  ) 0100 

004  K812503D  (04  ) 0010 

005  K812503M  (M  ) 0001 


SUBJ0063 

SINCE  9TH  GRADE.  HOW  MUCH  BIOLOGY? 


K812503 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADE S /ASS E S SMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0064 

SINCE  9TH  GRADE.  HOW  MUCH  LIFE  SCIENCE  (NOT  BIO)'’ 

N12 

K812504  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K812504A  (01  ) 0000 

002  K812504B  (02  ) 1000 

003  K812504C  (03  ) 0100 

004  K812504D  (04  ) 0010 

005  K812504M  (M  ) 0001 


MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID:  SUBJ0065 


2 

1 


2 

1 


3 

2 


5 

4 


5 

4 


5 

4 


5 

4 


ERIC 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


ERIC 


DESCRIPTION: 

SINCE  9TH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

K812505 

TYPE  OF  CONTRAST: 

CLASS 

001  K812505A  (01  ) 

0000 

002  K812505B  (02  ) 

1000 

003  K812505C  (03  ) 

0100 

004  K812505D  (04  ) 

0010 

005  K812505M  (M  ) 

0001 

CONDITIONING  VARIABLE  ID: 

SUBJ0066 

DESCRIPTION: 

SINCE  9TH 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID; 

K8 12506 

TYPE  OF  CONTRAST: 

CLASS 

001  K812506A  (01  ) 

0000 

002  K812506B  (02  ) 

1000 

003  K812506C  (03  ) 

0100 

004  K812506D  (04  ) 

1 0010 

005  K812506M  (M  ) 

1 0001 

CONDITIONING  VARIABLE  ID; 

SUBJ0067 

DESCRIPTION: 

SINCE  9TH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

K812507 

TYPE  OF  CONTRAST; 

CLASS 

001  K812507A  (01  ) 

1 0000 

002  K812507B  (02  ) 

1 1000 

003  K812507C  (03  ] 

1 0100 

004  K812507D  (04  ] 

1 0010 

005  K812507M  (M  I 

1 0001 

CONDITIONING  VARIABLE  ID: 

SUBJ0068 

DESCRIPTION: 

SINCE  9TH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

K812508 

TYPE  OF  CONTRAST: 

CLASS 

001  K812508A  (01  1 

1 0000 

002  K812508B  (02  ) 

1 1000 

003  K812508C  (03  1 

1 0100 

004  K812508D  (04  1 

1 0010 

005  K812508M  (M  ) 

1 0001 

CONDITIONING  VARIABLE  ID: 

SUBJ0069 

DESCRIPTION: 

SINCE  9TH 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

K812509 

TYPE  OF  CONTRAST: 

CLASS 

001  K812509A  (01  1 

1 0000 

002  K812509B  (02  ! 

1 1000 

003  K812509C  (03  1 

1 0100 

004  K812509D  (04  1 

1 0010 

005  K812509M  (M  1 

1 0001 

CONDITIONING  VARIABLE  ID: 

SUBJ0070 

DESCRIPTION: 

SINCE  9TH 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N12 

NAEP  ID: 

K812510 

TYPE  OF  CONTRAST: 

CLASS 

001  K812510A  (01 

) 0000 

002  K812510B  (02 

) 1000 

003  K812510C  (03 

) OlOO 

004  K812510D  (04 

) OOlO 

005  K812510M  (M 

) 0001 

CONDITIONING  VARIABLE  ID: 

SUBJ0071 

DESCRIPTION: 

ENROLLED 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

N12 

NAEP  ID; 

K812601 

TYPE  OF  CONTRAST; 

CLASS 

001  K812601Y  (01 

) 00 

002  K812601N  (02 

) 10 

003  K812601M  (M 

) 01 

BEST  CC,  . , 

HOW  MUCH  CHEMISTRY? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

more  than  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

MORE  THAN  1 YEAR 
1 YEAR 

LESS  THAN  1 YEAR 

NONE 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

NO 

missing 


•ilABLE 


-62 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0072 

DONE  SCI  INVEST/ PROJECTS  IN  SCHOOL  1 WK  OR  MORE? 
N12 


K812701 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K812701Y  (01  ) 00 

002  K812701N  (02  ) 10 

003  K812701M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SUBJ0073 

HOW  OFTEN  ANALYZE  OWN  SCI  DATA,  FORM  CONCLUS IONS'’ 
N12 


K811616 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  K811616A  (01  ) 0000 

002  K811616B  (02  ) 1000 

003  K811616C  (03  ) 0100 

004  K811616D  (04  ) 0010 

005  K811616M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE  A MTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0003 

DESCRIPTION:  BEST  DESCRIBES  HOW  4TH  GR  ARE  ORGANIZED? 


GRADES/ASSESSMENTS:  N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C030901 

TYPE  OF  CONTRAST:  CLASS 

001  C030901A  (01  ) 000 

002  C030901B  (02  ) 100 

003  C030901C  (03  ) 010 

004  C030901M  (M  ) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SELF  CONTAINED 
DEPARTMENTALIZED 
REGROUPED 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0004 

4TH  GRADERS  ASSIGNED  BY  ABILITY/ ACHIEVEMENT  LEVEL? 
N04 


C037101 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037101A  (01  ) 0000 

002  C037101B  (02  ) 1000 

003  C037101C  (03  ) 0100 

004  C037101N  (04  ) 0010 

005  C037101M  (M  ) 0001 


YES,  BY  MATH  ABILITY 
YES,  READING  ABILITY 
YES,  GENERAL  ABILITY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0005 

HOW  OFTEN  IS  4TH-GRADER  INSTRUCTED  IN  MATH? 
N04 


C031212 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  C031212A  (01  ) 00000 

002  C031212B  (02  ) 10000 

003  C031212C  (03  ) 01000 

004  C031212D  (04  ) 00100 

005  C031212E  (05  ) 00010 

006  C031212M  (M  ) 00001 


EVERY  DAY 

THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SCHL0006 

HOW  OFTEN  IS  4TH-GRADER  INSTRUCTED  IN  SCIENCE? 
N04 


C031205 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C031205A  (01  ) 00000 

002  C031205B  (02  ) 10000 

003  C031205C  (03  ) 01000 

004  C031205D  (04  ) 00100 

005  C031205E  (05  ) 00010 

006  C031205M  (M  ) 00001 


EVERY  DAY 

THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0007 

HOW  OFTEN  IS  4TH -GRADER  INSTRUCTED  IN  READING? 
N04 


C031213 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C031213A  (01  ) 00000 

002  C031213B  (02  ) 10000 


EVERY  DAY 

THREE  OR  FOUR/WEEK 


3 

2 


5 

4 


4 

3 


5 

4 


6 

5 


6 

5 


6 

5 


6 


r 

O 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


003 

C031213C 

(03 

) 01000 

004 

C031213D 

(04 

) 00100 

005 

C031213E 

(05 

) 00010 

006 

C031213M 

(M 

) 00001 

ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0008 

HOW  OFTEN  IS  4TH-GRADER  INSTRUCTED  IN  ARTS? 

N04 

COS  1214  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


6 

5 


001 

C031214A 

(01 

) 00000 

002 

C031214B 

(02 

) 10000 

003 

C031214C 

(03 

) 01000 

004 

C031214D 

(04 

) 00100 

005 

C031214E 

(05 

) 00010 

006 

C031214M 

(M 

) 00001 

EVERY  DAY 
THREE  OR  FOUR/WEEK 
ONCE  OR  TWICE  A WEEK 
LESS  THAN  ONCE/WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0009 

HAS  MATH  BEEN  IDENTIFIED  AS  A PRIORITY? 

N04,  NOS,  SOS 

C031603  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


3 

2 


001 

C031603Y 

(01 

) 00 

002 

C031603N 

(02 

) 10 

003 

C031603M 

(M 

) 01 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOOlO 

HAS  SCIENCE  BEEN  IDENTIFIED  AS  A PRIORITY? 

N04,  NOS,  SOS 

C031607  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


3 

2 


001 

C031607Y 

(01 

) 00 

002 

C031607N 

(02 

) 10 

003 

C031607M 

(M 

) 01 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHLOOll 

DESCRIPTION:  HAS  READING  BEEN  IDENTIFIED  AS  A PRIORITY? 


GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04 

NAEP  ID: 

C031601 

TYPE  OF  CONTRAST: 

CLASS 

001  C031601Y  (01 

) 00 

002  C031601N  (02 

) 10 

003  C031601M  (M 

) 01 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  3 

NUMBER  OF  INDEPENDENT  CONTRASTS:  2 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0012 

HAS  ARTS  BEEN  IDENTIFIED  AS  A PRIORITY? 

N04,  NOS,  SOS 

C031610  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


3 

2 


001 

C031610Y 

(01 

) 00 

002 

C031610N 

(02 

) 10 

003 

C031610M 

(M 

) 01 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 

SCHL0013 

DESCRIPTION: 

HAS  SUBJECT  INTEGRATION 

BEEN  A PRIORITY? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS 

NAEP  ID: 

C031606 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

001  C031606Y  (01  ) 

1 00 

YES 

002  C031606N  (02  ) 

1 10 

NO 

003  C031606M  (M  ) 

1 01 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0014 

DESCRIPTION: 

COMPUTERS 

AVAILABLE  ALL 

THE  TIME  IN  CLASSROOM? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS,  N12 

NAEP  ID: 

C035701 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

2 

001  C035701Y  (01  1 

1 00 

YES 

002  C035701N  (02  1 

1 10 

NO 

003  C035701M  (M  1 

1 01 

MISSING 

CONDITIONING  VARIABLE  ID; 

SCHL0015 

DESCRIPTION: 

COMPUTERS 

GROUPED  IN  SEPARATE  LAB  AND  AVAILABLE? 

ERIC  BEST  COPY  AVAILABLE  62C 


575 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS 

NAEP  ID: 

C035702 

TYPE  OF  CONTRAST: 

CLASS 

001  C035702Y  (01 

) 00 

002  C035702N  (02 

) 10 

003  C035702M  (M 

) 01 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0016 

COMPUTERS  AVAILABLE  TO  BRING  TO  ROOM  WHEN  NEEDED? 

N04,  NOS,  SOS,  N12 

C035703  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C035703Y 

(01 

) 00 

002 

C035703N 

(02 

) 10 

003 

C035703M 

(M 

) 01 

YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0017 

SCHOOL  W/  SPECIAL  FOCUS  ON  MATH? 

N04,  NOS,  SOS,  N12 

C037201  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037201Y  (01  ) 0 

002  C037201M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHLOOIS 

SCHOOL  W/  SPECIAL  FOCUS  ON  SCIENCE? 

N04,  NOS,  SOS,  N12 

C037202  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  C037202Y  (01  ) 0 

002  C037202M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0019 

SCHOOL  W/  SPECIAL  FOCUS  ON  MATH? 

N04 

C037207  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037207Y  (01  ) 0 

002  C037207M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0020 

SCHOOL  W/  SPECIAL  FOCUS  ON  ARTS? 

N04,  NOS,  SOS,  N12 

C037204  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037204Y  (01  ) 0 

002  C037204M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0021 

SCHOOL  W/  SPECIAL  FOCUS  ON  OTHER? 

N04,  NOS,  SOS,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


C037205 

CLASS 


001  C037205Y  (01  ) 0 

002  C037205M  (M  ) 1 


YES  ' 
MISSING 


r 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SCHL0022 

SCHOOL  NOT  A SPECIAL  FOCUS  SCHOOL? 

N04,  NOS,  SOS,  N12 

C037206  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037206Y  (01  ) 0 

002  C037206M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

grades/assessments ; 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SCHL0023 

SCHOOL  FOLLOW  DiSTRiCT/STATE  MATH  CURRICULUM? 

N04,  NOS,  SOS,  N12 

C037301  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


3 

2 


3 

2 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


O 6 


6 


Table  G-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  C037301Y  (01  ) 

002  C037301M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0024 

SCHOOL  FOLLOW  DiSTRiCT/STATE  SCIENCE  CURRICULUM? 
N04,  NOS,  SOS,  N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037302  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037302Y  (01  ) 

002  C037302M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0025 

SCHOOL  FOLLOW  DiSTRiCT/STATE  READING  CURRICULUM? 
N04 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037303  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037303Y  (01  ) 

002  C037303M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 

SCHL0026 

SCHOOL  FOLLOW  DISTRICT/STATE  ARTS  CURRICULUM? 
N04,  NOS,  SOS,  N12 

NAEP  ID; 

TYPE  OF  CONTRAST; 

C037304  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037304Y  (01  ) 

002  C037304M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES / ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

SCHL0027 

SCHOOL  FOLLOW  DISTRICT/STATE  FOR  NONE  OF  ABOVE? 
N04,  NOS,  SOS,  N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037305  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  . NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  C037305Y  (01  ) 

002  C037305M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL002S 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  MATH? 
N04 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037401Y  (01  ) 

002  C037401M  (M  ) 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0029 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  SCIENCE? 
N04 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037402  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037402Y  (01  ] 

002  C037402M  (M  1 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0030 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  READING? 
N04 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037403  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037403Y  (01 

002  C037403M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0031 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  ARTS? 
N04 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037404  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037404Y  (01 

002  C037404M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS: 
CONDITIONING  VAR  LABEL: 

SCHL0032 

SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  OTHER? 
NO  4 

BEST  COPY  AVAILABLE 


628 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C037405Y  (01 

002  C037405M  (M 


C037405 

CLASS 


) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0033 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C037406Y  (01 

002  C037406M  (M 


SCHOOL  SPONSER  4TH  GRDS  FIELD  TRIP  FOR  NONE  ABOVE? 
N04 


C037406 

CLASS 


) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0034 

DESCRIPTION:  4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  MATH? 

GRADES/ ASSESSMENTS : N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C037501  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST:  CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037501Y  (01 

002  C037501M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0035 

DESCRIPTION:  4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  SCIENCE’ 

GRADES /ASSESSMENTS:  N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C037502 

TYPE  OF  CONTRAST:  CLASS 


001  C037502Y  (01 

002  C037502M  (M 


) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0036 

DESCRIPTION:  4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  READING’ 

GRADES /ASSESSMENTS:  N04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C037503  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST:  CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037503Y  (01 

002  C037503M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0037 

DESCRIPTION:  4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  ARTS? 

GRADES /ASSESSMENTS:  n04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C037504 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037504Y  (01 

002  C037504M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0038 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C037505Y  (01 

002  C037505M  (M 


4TH  GRADERS  IN  EXTRACURR  ACTS  FOR  NONE  OF  ABOVE? 
N04 


C037505 

CLASS 


) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0039 

DESCRIPTION:  4TH  GRADERS  IN  SUMMER  PROGRAMS  IN  MATH? 

GRADES /ASSESSMENTS:  n04 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C037601 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037601Y  (01 

002  C037601M  (M 


) 0 
) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0040 

DESCRIPTION:  4TH  GRADERS  IN  SUMMER  PROGRAMS  IN  SCIENCE? 


GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C037602Y  (01 

002  C037602M  (M 


NO  4 


C037602 

CLASS 


) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0041 


O 1 

ERIC 


6.3 


Table  C*5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


ERIC 


DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


4TH  GRADERS  IN  SUMMER  PROGRAMS  IN  READING? 
N04 


C037603  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037603Y  {01  ) 0 

002  C037603M  {M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0042 

4TH  GRADERS  IN  SUMMER  PROGRAMS  IN  ARTS? 

N04 

C037604  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037604Y  (01  ) 0 

002  C037604M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0043 

4TH  GRADERS  IN  SUMMER  PROGRAMS  IN  NONE  OF  ABOVE? 

N04 

C037605  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C037605Y  (01  ) 0 

002  C037605M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0044 

WHICH  BEST  DESCRIBES  PRIMARY  WAY  LIBRARY  STAFFED? 

N04,  NOS,  SOS,  N12' 

C036601  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C036601A 

(01 

) 0000 

002 

C036601B 

(02 

) 1000 

003 

C036601C 

(03 

) 0100 

004 

C036601D 

(04 

) 0010 

005 

C036601M 

(M 

) 0001 

NO  LIBRARY  IN  SCHOOL 
LIBRARY-NO/VOL  STAFF 
PART-TIME  STAFF 
FULL-TIME  STAFF 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0045 

INVOLVE  PARENTS  AS  AIDES  IN  CLASSROOM? 

N04,  NOS,  SOS,  N12 

C032207  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C032207A 

(01 

) 000 

002 

C032207B 

(02 

) 100 

003 

C032207N 

(03 

) 010 

004 

C032207M 

(M 

) 001 

YES,  ROUTINELY 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0046 

DESCRIPTION:  HAVE  PARENTS  REVIEW/ SIGN  HOMEWORK? 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS 

NAEP  ID: 

C032209 

TYPE  OF  CONTRAST: 

CLASS 

001  C032209A  (01 

) 000 

002  C032209B  (02 

) 100 

003  C032209N  (03 

) 010 

004  C032209M  (M 

) 001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES,  ROUTINELY 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0047 

DESCRIPTION:  ASSIGN  HOMEWORK  STUDENTS  DO  WITH  PARENTS? 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS 

NAEP  ID: 

C032210 

TYPE  OF  CONTRAST: 

CLASS 

001  C032210A  (01 

) 000 

002  C032210B  (02 

) 100 

003  C032210N  (03 

) 010 

004  C032210M  (M 

) 001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES,  ROUTINELY 
YES,  OCCASIONALLY 
NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0048 

DESCRIPTION:  HAVE  A PARENT  VOLUNTEER  PROGRAM? 


GRADES /ASSESSMENTS : 

N04,  NOS,  SOS,  N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C032211 

TYPE  OF  CONTRAST: 

CLASS 

001  C032211A  (01 

) 000 

002  C032211B  (02 

) 100 

BEST 

COPY  AVAILABLE 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES,  ROUTINELY 
YES,  OCCASIONALLY 


2 

1 


2 

1 


2 

1 


4 

3 


4 

3 


4 

3 


4 

3 


579 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


003  C032211N  (03 

004  C032211M  (M 


) 010 
) 001 


NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0049 


DESCRIPTION; 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  C037701A  (01 

002  C037701B  (02 

003  C037701C  (03 

004  C037701D  (04 

005  C037701M  (M 


WHAT  % OF  PARENTS  IN  PARENT-TEACHER  ORGS? 
N04,  NOS,  SOS,  N12 


C037701 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID;  SCHL0050 


DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  C037702A  (01 

002  C037702B  (02 

003  C037702C  (03 

004  C037702D  (04 

005  C037702M  (M 


WHAT  % OF  PARENTS  IN  OPEN  HOUSE/BACK  SCHOOL  NIGHT? 
N04,  NOS,  SOS,  N12 


C037702 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID;  SCHL0051 


DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  C037703A  (01 

002  C037703B  (02 

003  C037703C  (03 

004  C037703D  (04 

005  C037703M  (M 


WHAT  % OF  PARENTS  IN  PARENT-TEACHER  CONFERENCES? 
N04,  NOS,  SOS,  N12 


C037703 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID;  SCHL0052 


DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  C037704A  (01 

002  C037704B  (02 

003  C037704C  (03 

004  C037704D  (04 

005  C037704M  (M 


WHAT  % PARENTS  INVOLVED  MAKING  CURRICULUM  DECISION 
N04,  NOS,  SOS,  N12 


C037704 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID;  SCHL0053 


DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  C037705A  (01 

002  C037705B  (02 

003  C037705C  (03 

004  C037705D  (04 

005  C037705M  (M 


WHAT  % OF  PARENTS  IN  VOLUNTEER  PROGRAMS? 
N04,  NOS,  SOS,  N12 


C037705 

CLASS 


) 0000 


) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

0-25% 

26-50% 

51-75% 

76-100% 

MISSING 


CONDITIONING  VARIABLE  ID;  SCHL0054 


DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 

001  C032402A  (01 

002  C032402B  (02 

003  C032402C  (03 

004  C032402D  (04 

005  C032402M  (M 


IS  STUDENT  ABSENTEEISM  A PROBLEM  IN  YOUR  SCHOOL? 
N04,  NOS,  SOS,  N12 


C032402 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS ; 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0055 

IS  STUDENT  TARDINESS  A PROBLEM  IN  YOUR  SCHOOL? 
N04,  NOS,  SOS,  N12 


C032401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


^ 6.1 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001 

C032401A 

(01 

) 0000 

SERIOUS 

002 

C032401B 

(02 

) 1000 

MODERATE 

003 

C032401C 

(03 

) 0100 

MINOR 

004 

C032401D 

(04 

) 0010 

NOT  A PROBLEM 

005 

C032401M 

(M 

) 0001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP.  ID: 

TYPE  OF  CONTRAST: 


SCHL0056 

ARE  PHYSICAL  CONFLICTS  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  NOB,  SOB,  N12 

C032404  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001 

C032404A 

(01 

.)  0000 

002 

C032404B 

(02 

) 1000 

003 

C032404C 

(03 

) 0100 

004 

C032404D 

(04 

) 0010 

005 

C032404M 

(M 

) 0001 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0057 

IS  TEACHER  ABSENTEEISM  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  NOB,  SOB,  N12 

C032406  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001 

C032406A 

(01 

) 0000 

002 

C032406B 

(02 

) 1000 

003 

C032406C 

(03 

) 0100 

004 

C032406D 

(04 

) 0010 

005 

C032406M 

(M 

) 0001 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL005B 

ARE  RACE/CULT.  CONFLICTS  A PROBLEM  IN  YOUR  SCHOOL? 

N04,  NOB,  SOB,  N12 

CO 32 407  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001 

C032407A 

(01 

) 0000 

002 

C032407B 

(02 

) 1000 

003 

C032407C 

(03 

) 0100 

004 

C032407D 

(04 

) 0010 

005 

C032407M 

(M 

) 0001 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 

SCHL0059 

DESCRIPTION: 

IS  STUDENT  HEALTH  A PROBLEM  IN  YOUR  SCHOOL? 

GRADES /ASSESSMENTS : 

N04,  NOB 

, SOB, 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C032408 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C032408A  (01  ) 

1 0000 

SERIOUS 

002  C032408B  (02  ) 

1 1000 

MODERATE 

003  C032408C  (03  ) 

1 0100 

MINOR 

004  C032408D  (04^  ) 

1 0010 

NOT  A PROBLEM 

005  C03240BM  (M  ’ ) 

f 0001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0060 

DESCRIPTION: 

IS  LACK 

OF  PARENT  INVLMT  A PROBLEM  IN  YOUR  SCHOOL? 

GRADES /ASSESSMENTS : 

N04,  NOB 

, SOB, 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C032409 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C032409A  (01  1 

1 0000 

SERIOUS 

002  C032409B  (02  1 

\ 1000 

MODERATE 

003  C032409C  (03  1 

1 0100 

MINOR 

004  C032409D  (04  1 

1 0010 

NOT  A PROBLEM 

005  C032409M  (M  1 

1 0001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0061 

DESCRIPTION: 

IS  STUD 

USE  OF 

ALCOHOL  A PROBLEM  IN  YOUR  SCHOOL? 

GRADES /ASSESSMENTS : 

N04,  NOB 

, SOB, 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C032410 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C032410A  (01 

) 0000 

SERIOUS 

002  C032410B  (02 

1 1000 

MODERATE 

003  C032410C  (03 

) 0100 

MINOR 

004  C032410D  (04 

) 0010 

NOT  A PROBLEM 

005  C032410M  (M 

) 0001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


SCHL0062 

IS  STUDENT  TOBACCO  USE  A PROBLEM  IN  YOUR  SCHOOL? 


632 


BEST  COPY  AVAILABLE 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C032411A  (01 

002  C032411B  (02 

003  C032411C  (03 

004  C032411D  (04 

005  C032411M  (M 


N04,  NOS,  SOS,  N12 

C032411 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0063 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

TYPE  OF  CONTRAST: 


IS  STUDENT  DRUG  USE  A PROBLEM  IN  YOUR  SCHOOL? 
N04,  NOS,  SOS,  N12 


C032412 

CLASS 


001  C032412A 

002  C032412B 

003  C032412C 

004  C032412D 

005  C032412M 


(01 

(02 

(03 

(04 

(M 


) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0064 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C032413 

TYPE  OF  CONTRAST:  CLASS 


ARE  GANG  ACTIVITIES  A PROBLEM  IN  YOUR  SCHOOL? 
N04,  NOS,  SOS,  N12 


001  C032413A  (01 

002  C032413B  (02 

003  C032413C  (03 

004  C032413D  (04 

005  C032413M  (M 


) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0065 

DESCRIPTION: 

GRADES/ ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C032414 

TYPE  OF  CONTRAST:  CLASS 

001  C032414A  (01  ) 0000 

002  C032414B  (02  ) 1000 

003  C032414C  (03  ) 0100 

004  C032414D  (04  ) 0010 

005  C032414M  (M  ) 0001 

CONDITIONING  VARIABLE  ID:  SCHL0066 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C032415 

TYPE  OF  CONTRAST:  CLASS 

001  C032415A  (01  ) 0000 

002  C032415B  (02  ) 1000 

003  C032415C  (03  ) 0100 

004  C032415D  (04  ) 0010 

005  C032415M  (M  ) 0001 


IS  STUDENT  MISBEHAVIOR  A PROBLEM  IN  YOUR  SCHOOL? 
N04,  NOS,  SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


IS  STUDENT  CHEATING  A PROBLEM  IN  YOUR'  SCHOOL? 
N04,  NOS,  SOS,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 

SERIOUS 

MODERATE 

MINOR 

NOT  A PROBLEM 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C032502A  (01 

002  C032502B  (02 

003  C032502C  (03 

004  C032502D  (04 

005  C032502M  (M 


SCHL0067 

IS  TEACHER  MORALE  POS . OR  NEG.? 
N04,  NOS,  SOS,  N12 

C032502 

CLASS 

) 0000 
) 1000 
) 0100 
) 0010 
) 0001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL006S 


DESCRIPTION: 


ERIC 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  N08 

NAEP  ID: 

C032503 

TYPE  OF  CONTRAST: 

CLASS 

001  C032503A  (01 

) 0000 

002  C032503B  (02 

) 1000 

003  C032503C  (03 

) 0100 

004  C032503D  (04 

) 0010 

'82 

1 

6.  3 

are  student  attitudes  to  academics  POS.  OR  NEG.? 
S,  N12 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


005  C032503M  (M 


0001 


MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0069 

DESCRIPTION:  IS  PARENT  SUPPORT  FOR  ACHIEVEMENT  POS . OR  NEG.? 


GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS, 

N12 

NAEP  ID: 

C032505 

TYPE  OF  CONTRAST: 

CLASS 

001  C032505A  (01  ] 

1 0000 

002  C032505B  (02  ! 

1 1000 

003  C032505C  (03  1 

1 0100 

004  C032505D  (04  ! 

1 0010 

005  C032505M  (M  ! 

1 0001 

CONDITIONING  VARIABLE  ID: 

SCHL0070 

DESCRIPTION: 

IS  REGARD 

FOR 

SCHOOL 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS, 

N12 

NAEP  ID: 

C032506 

TYPE  OF  CONTRAST: 

CLASS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


PROPERTY  POS.  OR  NEG.? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C032506A  (01 

002  C032506B  (02 

003  C032506C  (03 

004  C032506D  (04 

005  C032506M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


) 0000 
) 1000 
) 0100 
) 0010 
) 0001 

SCHL0071 

% ABSENT  ON  AVERAGE  DAY? 
N04,  NOS,  SOS,  N12 

C033601 

CLASS 


VERY  POSITIVE 
SOMEWHAT  POSITIVE 
SOMEWHAT  NEGATIVE 
VERY  NEGATIVE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C033601A 

(01 

) 0000 

002 

C033601B 

(02 

) 1000 

003 

C033601C 

(03 

) 0100 

004 

C033601D 

(04 

) 0010 

005 

C033601M 

(M 

) 0001 

0-2% 

3-5% 

6-10% 

MORE  THAN  10% 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0072 

WHAT  % OF  TEACHERS  ABSENT  ON  GIVEN  DAY? 
N04,  NOS,  SOS,  N12 


C036501 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C036501A 

(01 

) 0000 

0-2% 

002 

C036501B 

(02 

) 1000 

3-5% 

003 

C036501C 

(03 

) 0100 

6-10% 

004 

C036501D 

(04 

) 0010 

MORE  THAN 

005 

C036501M 

(M 

) 0001 

MISSING 

5 

4 


5 

4 


5 

4 


5 

4 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001 

C037S01A 

(01 

) 00000000 

002 

C037S01B 

(02 

) 10000000 

003 

C037S01C 

(03 

) 01000000 

004 

C037S01D 

(04 

) 00100000 

005 

C037S01E 

(05 

) 00010000 

006 

C037S01F 

(06 

) 00001000 

007 

C037S01G 

(07 

) 00000100 

OOS 

C037S01H 

(OS 

) 00000010 

009 

C037S01M 

(M 

) 00000001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

9S-100% 

95-97% 

90-94% 

S0-S9% 

70-79% 

60-69% 

50-59% 

LESS  THAN  50% 

MISSING 


SCHL0073 

% OF  STUDS  EROLLED  AT  START  OF  YR  EROLLED  AT  END? 
N04,  NOS,  SOS,  N12 

C037S01 
CLASS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0074 

% OF  4TH  GRADERS  HELD  BACK  & REPEATING  4TH  GRADE? 

N04 

C037901  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

C037901A 

(01 

) 00000 

0% 

002 

C037901B 

(02 

) 10000 

1-2% 

003 

C037901C 

(03 

) 01000 

3-5% 

004 

C037901D 

(04 

) 00100 

6-10% 

005 

C037901E 

(05 

) 00010 

MORE  THAN 

006 

C037901M 

(M 

) 00001 

MISSING 

9 

S 


6 

5 


CONDITIONING  VARIABLE  ID:  SCHL0075 

DESCRIPTION:  % OF  FULL  TIME  TEACHERS  LEFT  BEFORE  END  OF  YR? 


634 


583 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C038001A  (01 

002  C038001B  (02 

003  C038001C  (03 

004  C038001D  (04 

005  C038001E  (05 

006  C038001F  (06 

007  C038001G  (07 

008  C038001M  (M 


N04,  N08,  S08,  N12 

C038001 

CLASS 

) 0000000 
) 1000000 
) 0100000 
) 0010000 
) 0001000 
) 0000100 
) 0000010 
) 0000001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

0% 

1-2% 

3-5% 

6-10% 

11-15% 

16-20% 

MORE  THAN  20% 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0076 

IS  SCHOOL  IN  NATIONAL  SCHOOL  LUNCH  PROGRAM? 

N04,  N08,  S08,  N12 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


C038301 

CLASS 


001  C038301Y  (01  ) 00 

002  C038301N  (02  ) 10 

003  C038301M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0077 

DESCRIPTION:  SCHOOL  RECEIVE  CHAP  1/TITLE  1 FUNDING? 

GRADES /ASSESSMENTS:  N04 , N08,  S08,  N12 

CONDITIONING  VAR  LABEL: 

C038801 
CLASS 


NAEP  ID: 

TYPE  OF  CONTRAST: 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C038801Y  (01  ) 00 

002  C038801N  (02  ) 10 

003  C038801M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0078 

DID  PRINCIPAL  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  N08,  S08,  N12 

C034101  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034101Y  (01  . ) 0 

002  C034101M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0079 

DID  HEADMASTER/HEADMISTRESS  FILL  OUT  QUESTIONNAIRE 
N04,  N08,  S08,  N12 

C034102  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034102Y  (01  ) 0 

002  C034102M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0080 

DID  HEAD  TEACHER  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  N08,  S08,  N12 

C034103  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034103Y  (01  ) 0 

002  C034103M  (M  ) 1 


YES 

MISSING 


SCHL0081 

DID  VICE  PRINCIPAL  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  N08,  S08,  N12 

C()34104  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C034104Y  (01  ) 0 

002  C034104M  (M  ) 1 

CONDITIONING  VARIABLE  ID;  SCHL0082 

DESCRIPTION:  DID  COUNSELOR  FILL  OUT  THIS  QUESTIONNAIRE 

GRADES /ASSESSMENTS : N04,  N08,  S08,  N12 


CONDITIONING  VAR  LABEL: 
NAEP  ID: 

C034105 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C034105Y  (01  ) 

1 0 

YES 

002  C034105M  (M  ) 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0083 

YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


8 

7 


3 

2 


3 

2 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


DID  CURRICULUM  COORD  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  NOS,  SOS,  N12 

C034106  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034106Y  (01  ) 0 

002  C034106M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00S4 

DID  TEACHER  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  NOS,  SOS,  N12 

C034107  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034107Y  (01  ) 0 

002  C034107M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00S5 

DID  SECRETARY  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  NOS,  SOS,  N12 

C03410S  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C03410SY  (01  ) 0 

002  C03410SM  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL00S6 

DID  OTHER  PERSON  FILL  OUT  THIS  QUESTIONNAIRE 
N04,  NOS,  SOS,  N12 

C034109  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C034109Y  (01  ) 0 YES 

002  C034109M  (M  ) 1 MISSING 


CONDITIONING  VARIABLE  ID:  TCHROOOl 

DESCRIPTION:  WHAT  IS  YOUR  GENDER? 

GRADES /ASSESSMENTS:  N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


NAEP  ID:  T055901 

TYPE  OF  CONTRAST:  CLASS 


001 

T055901A 

(01 

) 00 

MALE 

002 

T055901B 

(02 

) 10 

FEMALE 

003 

T055901M 

(M 

) 01 

MISSING 

2 

1 


2 

1 


2 

1 


2 

1 


3 

2 


CONDITIONING  VARIABLE  ID:  TCHR0002 

DESCRIPTION:  WHICH  BEST  DESRIBES  YOU? 


GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS 

NAEP  ID: 

T056001 

TYPE  OF  CONTRAST: 

CLASS 

001  T056001A  (01  ] 

1 000000 

002  T056001B  (02  ] 

1 100000 

003  T056001C  (03  1 

1 010000 

004  T056001D  (04  1 

1 001000 

005  T056001E  (05  1 

1 000100 

006  T056001F  (06  1 

1 000010 

007  T056001M  (M  ] 

1 000001 

CONDITIONING  VARIABLE  ID: 

TCHR0003 

DESCRIPTION: 

YEARS  TAUGHT 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS 

NAEP  ID: 

T040301 

TYPE  OF  CONTRAST: 

CLASS 

001  T040301A  (01 

) 00000 

002  T040301B  (02 

) 10000 

003  T040301C  (03 

) 01000 

004  T040301D  (04 

) 00100 

005  T040301E  (05 

) 00010 

006  T040301M  (M 

) 00001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

WHITE 

BLACK 

HISPANIC 

ASIAN/PACIFIC  AMERIC 
AMER  IND/ALASKA  NATV 
OTHER 
MISSING  . 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 YEARS  OR  LESS 
3-5  YEARS 
6-10  YEARS 
11-24  YEARS 
25  YEARS  OR  MORE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0004 

HOW  MANY  YRS  TOTAL  YOU  TAUGHT  MATH? 

N04 

T056101  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056101A  (01  ) 00000 


2 YEARS  OR  LESS 


7 

6 


6 

5 


6 

5 


636 


585 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


002 

T056101B 

(02 

) 10000 

003 

T056101C 

(03 

) 01000 

004 

T056101D 

(04 

) 00100 

005 

T056101E 

(05 

) 00010 

006 

T056101M 

(M 

) 00001 

3-5  YEARS 
6-10  YEARS 
11-24  YEARS 
25  YEARS  OR  MORE 
MISSING 


CONDITIONING 

VARIABLE  ID: 

TCHR0005 

DESCRIPTION: 

HOW  MANY  YRS  TOTAL 

GRADES /ASSESSMENTS : 

N04 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

T056102 

TYPE  OF  CONTRAST: 

CLASS 

001  T056102A 

(01 

1 00000 

002  T056102B 

(02 

1 10000 

003  T056102C 

(03  ] 

1 01000 

004  T056102D 

(04  ] 

1 00100 

005  T056102E 

(05  ] 

1 00010 

006  T056102M 

(M  ] 

1 00001 

CONDITIONING 

VARIABLE  ID: 

TCHR0006 

DESCRIPTION: 

TYPE  TCHNG  CERT  IN 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

T056201 

TYPE  OF  CONTRAST: 

CLASS 

001  T056201A 

(01  • ) 

000000 

002  T056201B 

(02  ) 

100000 

003  T056201C 

(03  ) 

010000 

004  T056201D 

(04  ) 

001000 

005  T056201E 

(05  ) 

000100 

006  T056201F 

(06  ) 

000010 

007  T056201M 

(M  ) 

000001 

TAUGHT  SCIENCE? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 YEARS  OR  LESS 
3-5  YEARS 
6-10  YEARS 
11-24  YEARS 
25  YEARS  OR  MORE 
MISSING 


ST  IN  MAIN  FIELD? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ADVANCED  PROFESS I ONL 
REGULAR/ STANDARD  ST 
PROBATIONARY  STATE 
TEMPORARY/ PROVI S lONL 
OTHER  THAN  STATE  CRT 
NOT  HAVE  CERT  MAIN 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0007 

CERTIFICATION,  ELEMENTARY  OR  MIDDLE/ JUNIOR  HS  ED? 

N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T040501 

CLASS 


001  T040501Y  (01  ) 000 

002  T040501N  (02  ) 100 

003  T040501C  (03  ) 010 

004  T040501M  (M  ) 001 


YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROOOS 

DO  YOU  HAVE  CERTIFICATION  IN  ELEMENTARY  MATH? 

N04 

T040506  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040506Y  (01  ) 000 

002  T040506N  (02  ) 100 

003  T040506C  (03  ) 010 

004  T040506M  (M  ) 001 


YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0009 

DO  YOU  HAVE  CERTIFICATION  IN  JR  HIGH/ SEC  MATH? 

NO  4 

T040504  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040504Y  (01  ) 000 

002  T040504N  (02  ) 100 

003  T040504C  (03  ) 010 

004  T040504M  (M  ) 001 


YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROOlO 

CERTIFICATION,  ELEMENTARY  SCIENCE? 

N04,  NOS,  SOS 

T040507  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040507Y  (01  ) 000 

002  T040507N  (02  ) 100 

003  T040507C  (03  ) 010 

♦ 004  T040507M  (M  ) 001 


YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


TCHROOll 

CERTIFICATION,  MIDDLE/ JUNIOR  SCIENCE 
N04,  NOS,  SOS 


6 

5 


7 

6 


4 

3 


4 

3 


4 

3 


4 

3 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


NAEP  ID;  T040508 

TYPE  OF  CONTRAST;  CLASS 

001  T040508Y  (01  ) 000 

002  T040508N  (02  ) 100 

003  T040508C  (03  ) 010 

004  T040508M, (M  ) 001 

conditioning  variable  ID; 

DESCRIPTION: 

grades/assessments ; 
conditioning  VAR  LABEL: 

NAEP  ID; 

TYPE  OF  CONTRAST; 

001  T040505Y  (01 

002  T040505N  (02 

003  T040505C  (03 

004  T040505M  (M 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 

CONDITIONING  VAR  LABEL: 

NAEP  ID; 


TCHR0012 

CERTIFICATION,  OTHER 
N04,  N08,  S08 

T040505 

CLASS 

) 000 
) 100 
) 010 
) 001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

NO 

NOT  OFFERED  IN  STATE 
MISSING 


TCHR0013 

HIGHEST  ACADEMIC  DEGREE  YOU  HOLD? 
N04,  N08,  S08 

T056301 


ERIC 


TYPE  OF  CONTRAST; 

CLASS 

001  T056301A  (01 

) 0000000 

002  T056301B  (02 

) 1000000 

003  T056301C  (03 

) 0100000 

004  T056301D  (04 

) 0010000 

005  T056301E  (05 

) 0001000 

006  T056301F  (06 

) 0000100 

007  T056301G  (07 

) 0000010 

008  T056301M  (M 

) 0000001 

CONDITIONING  VARIABLE  ID: 

TCHR0014 

DESCRIPTION; 

EDUCATION 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 

N04,  N08, 

NAEP  ID: 

T040701 

TYPE  OF  CONTRAST: 

CLASS 

001  T040701Y  (01 

) 0 

002  T040701M  (M 

) • 1 

CONDITIONING  VARIABLE  ID 

: TCHR0015 

DESCRIPTION: 

ELMENT  ED 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  N08, 

NAEP  ID: 

T040706 

TYPE  OF  CONTRAST: 

CLASS 

001  T040706Y  (01 

) 0 

002  T040706M  (M 

) 1 

CONDITIONING  VARIABLE  ID 

; TCHR0016 

DESCRIPTION: 

SEC  ED  UNI 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

N04,  N08, 

NAEP  ID; 

T040707 

TYPE  OF  CONTRAST; 

CLASS 

001  T040707Y  (01 

) 0 

002  T040707M  (M 

) 1 

CONDITIONING  VARIABLE  ID 

i;  TCHR0017 

DESCRIPTION; 

WAS  YOUR 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

N04 

NAEP  ID: 

T040703 

TYPE  OF  CONTRAST; 

CLASS 

001  T040703Y  (01 

) 0 

002  T040703M  (M 

) 1 

CONDITIONING  VARIABLE  ID;  TCHR0018 

DESCRIPTION: 

WAS  YOUR 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 

NO  4 

NAEP  ID: 

T040704 

TYPE  OF  CONTRAST; 

CLASS 

001  T040704Y  (01 

) 0 

002  T040704M  (M 

) 1 

BEST  COPY 

AVAILAB 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

HIGH  SCHOOL  DIPLOMA 
ASSOC lATES / VOCAT lONL 
BACHELORS  DEGREE 
MASTER'S  DEGREE 
EDUCATION  SPECIALIST 
DOCTORATE 

PROFESSIONAL  DEGREE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT' CONTRASTS ; 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS ; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


4 

3 


4 

3 


8 

7 


2 

1 


2 

1 


2 

1 


2 

1 


658 


587 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040710Y  (01  ) 

002  T040710M  (M  ) 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040711Y  (01  ) 

002  T040711M  (M  ) 


TCHR0019 

SCIENCE  ED  UNDERGRAD  MAJOR 
N04,  NOS,  SOS 

T040710 

CLASS 

0 

1 

TCHR0020 

LIFE  SCIENCE  UNDERGRAD  MAJOR? 
N04,  NOS,  SOS 

T040711 

CLASS 

0 

1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0021 

PHYSICAL  SCIENCE  UNDERGRAD  MAJOR? 

N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T040712 

CLASS 


001  T040712Y  (01  ) 0 

002  T040712M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0022 

EARTH  SCIENCE  UNDERGRAD  MAJOR? 
N04,  NOS,  SOS 

T040713 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040713Y  (01  ) 0 

002  T040713M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0023 

SPECIAL  EDUCATION  UNDERGRAD  MAJOR 
N04,  NOS,  SOS 

T04070S  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T04070SY  (01  ) 0 

002  T04070SM  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0024 

BILINGUAL  ED/ESL  UNDERGRAD  MAJOR 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T040709 

CLASS 


001  T040709Y  (01  ) 0 

002  T040709M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040705Y  (01 

002  T040705M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040801Y  (01 

002  T040801M  (M 


TCHR0025 

OTHER  UNDERGRAD  MAJOR 
N04,  N08,  S08 

T040705 

CLASS 

) 0 
) 1 

TCHR0026 

EDUCATION  GRAD  MAJOR 
N04,  N08,  S08 

T040801 

CLASS 

) 0 
) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0027 

DESCRIPTION:  ELEMENTARY  ED  GRAD  MAJOR 

GRADES /ASSESSMENTS:  N04,  N08,  S08 

CONDITIONING  VAR  LABEL: 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


NAEP  ID:  T040807 

TYPE  OF  CONTRAST:  CLASS 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 
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Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  T040807Y  (01  ) 0 

002  T040807M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0028 

SECONDARY  ED  GRAD  MAJOR 
N04,.  N08,  S08 

T040808 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040808Y  (01  ) 0 

002  T040808M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0029 

WAS  YOUR  GRADUATE  MAJOR  MATHEMATICS? 

NO  4 

T040803  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  T040803Y  (01  ) 0 

002  T040803M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0030 

WAS  YOUR  GRADUATE  MAJOR  MATH  ED? 

N04 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T040804 

CLASS 


001  T040804Y  (01  ) 0 

002  T040804M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040814Y  (01 

002  T040814M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST : 

001  T040815Y  (01 

002  T040815M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0031 

SCIENCE  ED  GRAD  MAJOR? 

N04,  N08,  S08 

T040814 

CLASS 

) 0 
) 1 

TCHR0032 

LIFE  SCIENCE  GRAD  MAJOR? 

N04,  N08,  S08 

T040815 

CLASS 

) 0 
) 1 

TCHR0033 

PHYSICAL  SCIENCE  GRAD  MAJOR? 
N04,  N08,  S08 

T040816 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
■NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040816Y  (01  ) 0 

002  T040816M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0034 

EARTH  SCIENCE  GRAD  MAJOR? 
N04,  N08,  S08 

T040817 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040817Y  (01  ) 0 

002  T040817M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040809Y  (01 

002  T040809M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 


TCHR0035 

SPECIAL  ED  GRAD  MAJOR 
N04,  N08,  S08 

T040809 

CLASS 

) 0 
) 1 

TCHR0036 

BILINGUAL  GRAD  MAJOR 
N04,  N08,  S08 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040810Y  (01  ) 

002  T040810M  (M  ) 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T040811Y  (01  ) 

002  T040811M  (M  ) 


T040810 

CLASS 

0 

1 

TCHR0037 

ADMIN/SUPERVISION  GRAD  MAJOR 
N04,  N08,  S08 

T040811 

CLASS 

0 

1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR.  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0038 

CURRICULUM/ INSTRUCTION  GRAD  MAJOR? 
N04,  N08,  S08 


T040812 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040812Y  (01  ) 0 

002  T040812M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0039 

COUNSELING  GRAD  MAJOR? 
N04,  N08,  S08 

T040813 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T040813Y  (01  ) 0 

002  T040813M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0040 

OTHER  GRAD  MAJOR 
N04,  N08,  S08 

T040805 

CLASS 


001  T040805Y  (01  ) 0 

002  T040805M  (M  ) 1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0041 

NO  GRADUATE  STUDY 
N04,  N08,  S08 

T040806 

CLASS 


001  T040806Y  (01  ) 0 

002  T040806M  (M  ) 1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0042 

UNDERGRAD/GRAD  MINOR  STUDY-EDUCATION 
N04,  N08,  S08 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T056401 

CLASS 


001  T056401Y  (01  ) 0 

002  T056401M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0043 

UNDERGRAD/GRAD  MINOR  STUDY-ELEMENTARY  ED 
N04,  N08,  S08 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T056402 

CLASS 


001  T056402Y  (01  ) 0 

002  T056402M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0044 

UNDERGRAD /GRAD  MINOR  STUDY-SECONDARY  ED 
N04,  N08,  S08 

T056403  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056403Y  (01  ) 0 

002  T056403M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0045 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


ERIC 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION; 
grades/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


UNDERGRAD /GRAD  MINOR  STUDY-MATHEMATICS 
NO  4 


T056404 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056404Y  (01  ) 0 

002  T056404M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0046 

UNDERGRAD/GRAD  MINOR  STUDY-MATHEMATICS  ED 
N04,  NOB,  SOB 


T056405 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056405Y  (01  ) 0 

002  T056405M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TCHR0047 

UNDERGRAD/ GRAD  MINOR  STUDY- SCIENCE  ED 
N04,  NOB,  SOB 


T056413 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056413Y  (01  ) 0 ■ 

002  T056413M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 
GRADES/ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR004B 

UNDERGRAD /GRAD  MINOR  STUDY- LIFE  SCIENCE 
N04,  NOB,  SOB 


T056414 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056414Y  (01  ) 0 

002  T056414M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0049 

UNDERGRAD/GRAD  MINOR  STUDY- PHYSICAL  SCIENCE 
N04,  NOB,  SOB 


T056415 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056415Y  (01  ) 0 

002  T056415M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 
GRADES/ASSESSMENTS; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0050 

UNDERGRAD /GRAD  MINOR  STUDY- EARTH  SCIENCE 
N04 


T056416 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056416Y  (01  ) 0 

002  T056416M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0051 

UNDERGRAD/GRAD  minor  study-special  ED 
N04,  NOB,  SOB 


T056406 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056406Y  (01  ) 0 

002  T056406M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0052 

UNDERGRAD/GRAD  MINOR  STUDY- BILINGUAL  ED 
N04,  NOB,  SOB 


T056407 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T056407Y  (01  ) 0 

002  T056407M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHR0053 

UNDERGRAD/GRAD  MINOR  STUDY- ADMIN  U SUPERVISION 
N04,  NOB,  SOB 


T05640B 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T05640BY  (01  ) 0 


YES 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


002  T056408M  (M  ) 1 


MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0054 

UNDERGRAD /GRAD  MINOR  STUDY-CURRICULUM  & INSTRUC 
N04,  N08,  S08 


T056409 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056409Y  (01  ) 0 

002  T056409M  (M  ) 1 


YES 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0055 

UNDERGRAD /GRAD  MINOR  STUDY-COUNSELING 
N04,  N08,  S08 


T056410 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056410Y  (01  ) 0 

002  T056410M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0056 

UNDERGRAD /GRAD  MINOR  STUDY-OTHER 
N04,  N08,  S08 

T056411 
CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056411Y  (01  ) 0 

002  T056411M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0057 

UNDERGRAD/GRAD  MINOR  STUDY-NONE 
N04,  N08,  S08 

T056412 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056412Y  (01  ) 0 

002  T056412M  (M  ) 1 


YES 

MISSING 


CONDITIONING 

VARIABLE  ID: 

TCHR0058 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

LAST  YR,  HOW  MUCH  TIME 
N04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

T056501 

CLASS 

001  T056501A 

(01  ) 

' 00000 

002  T056501B 

(02  ) 

10000 

003  T056501C 

(03  ) 

01000 

004  T056501D 

(04  ) 

00100 

005  T056501E 

(05  ) 

00010 

006  T056501M 

(M  ) 

00001 

IN  MATH/MATH  ED  SEM/WRKSHP? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

LESS  THAN  6 HOURS 
6-15  HOURS 
16-35  HOURS 
MORE  THAN  35  HOURS 

missing 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0059 

LAST  YR,  HOW  MUCH  TIME  IN  SCI /SCI  ED  SEM/WRKSHPS’’ 
N04 


T058101 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T058101A  (01  ) 00000 

002  T058101B  (02  ) 10000 

003  T058101C  (03  ) 01000 

004  T058101D  (04  ) 00100 

005  T058101E  (05  ) 00010 

006  T058101M  (M  ) 00001 


NONE 

LESS  THAN  6 HOURS 
6-15  HOURS 
16-35  HOURS 
MORE  THAN  35  HOURS 
MISSING 


CONDITIONING 

VARIABLE  ID: 

TCHR0060 

DESCRIPTION: 

LAST  2 YRS 

GRADES /ASSESSMENTS : 

N04 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

T056601 

TYPE  OF  CONTRAST: 

CLASS 

001  T056601A 

(01  ) 

1 00000 

002  T056601B 

(02  ) 

10000 

003  T056601C 

(03  ) 

• 01000 

004  T056601D 

(04  ) 

00100 

005  T056601E 

(05  ) 

00010 

006  T056601M 

(M  ) 

00001 

ED  UNIV  COURSES? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

ONE 

TWO 

THREE 

FOUR  OR  MORE 
MISSING 


HOW  MANY  MATH /MATH 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 


TCHR0061 

LAST  2 YRS,  HOW  MANY  SCI/SCI  ED  UNIV  COURSES? 
N04 


Table  C?5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T058201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T05S201A  (01  ) 

00000 

NONE 

002  T05S201B  (02  ) 

10000 

ONE 

003  T05S201C  (03  ) 

01000 

TWO 

004  T05S201D  (04  ) 

00100 

THREE 

005  T05S201E  (05  ) 

00010 

FOUR  OR  MORE 

006  T05S201M  (M  ) 

00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0062 

DESCRIPTION: 

PAST  5 YRS,  TAKEN 

COURSES /IN 

PRO  DEVP- TELECOMM  USE 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T056701 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056701Y  (01  ) 

0 

YES 

002  T056701M  (M  ) 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0063 

DESCRIPTION: 

PAST  5 YRS,  TAKEN 

COURSES /IN 

PRO  DEVP -TECH  USE 

GRADES/ ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 

NAEP  ID: 

T056702 

• TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056702Y  (01  ] 

1 0 

YES 

002  T056702M  (M  ] 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0064 

DESCRIPTION: 

PAST  5 YRS,  TAKEN 

COURSES /IN 

PRO  DEVP -COOP  INSTRCT 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID; 

T056703 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056703Y  (01 

) 0 

YES 

002  T056703M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0065 

DESCRIPTION: 

PAST  5 YRS,  COURSES/ IN  PRO  DEVLP- INTERDISP  INSTRCT 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID; 

T056704 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST; 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS; 

001  T056704Y  (01 

) 0 

YES 

002  T056704M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0066 

PAST  5 YRS,  COURSES /IN  PRO  DEVLP- PORTFOLIO  ASSMNT 
N04,  NOS,  SOS 


T056705 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056705Y  (01 

002  T056705M  (M 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0067 

PAST  5 YRS,  COURSES /IN  PRO  DEVLP -PERF  BASED  ASSMNT 
N04,  NOS,  SOS 


T056706 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056706Y  (01 

002  T056706M  (M 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TCHR006S 

PAST  5 YRS,  COURSES/PRO  DEVLP-TEACH  HIGHORDER  THKG 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T056707 

CLASS 


001  T056707Y  (01  ) 0 

002  T056707M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0069 

PAST  5 YRS,  COURSES /PRO  DEVLP-TEACH  DIFF  CULT  BKGD 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T05670S 

CLASS 


6 

5 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


593 


BEST  COPY  AVAILABLE 


w L 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  T056708Y  (01  ) o 

002  T056708M  (M  ) l 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0070 

PAST  5 YRS,  COURSES/ PRO  DEVLP-TEACH  LEP  STUDENTS 
N04,  N08,  808 


T056709 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056709Y  (01  ) 0 

002  T056709M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0071 

PAST  5 YRS,  COURSES/ PRO  DEVLP-TEACH  SPEC  NEED  STDS 
N04,  NOS,  SOS 


T056710 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056710Y  (01  ) 0 

002  T056710M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0072 

PAST  5 YRS,  COURSES/PRO  DEVLP-CLASSRM' MNGMT/ORG 
N04,  NOS,  SOS 


T056711 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056711Y  (01  ) 0 

002  T056711M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0073 

PAST  5 YRS,  COURSES/PRO  DEVLP-OTHER  PROF  ISSUES 
N04,  NOS,  SOS 


T056712 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056712Y  (01  ) 0 

002  T056712M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0074 

PAST  5 YRS,  COURSES/ PRO  DEVLP-NONE  OF  ABOVE 
N04,  NOS,  SOS 


T056713 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T056713Y  (01  ) 0 

002  T056713M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0075 

AVAILABILITY  OF  RESOURCES 
N04,  NOS,  SOS 

T041201 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T041201A  (01  ) 0000 

002  T041201B  (02  ) 1000 

003  T041201C  (03  ) 0100 

004  T041201D  (04  ) 0010 

005  T041201M  (M  ) 0001 


GET  ALL  RESOURCES 
GET  MOST  RESOURCES 
GET  SOME  RESOURCES 
DONT  GET  RESOURCES 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0076 

ARE  CURRICULUM  SPECIALISTS  AVAILABLE  FOR  MATH’ 
N04 


T041302 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T041302Y  (01  ) 00 

002  T041302N  (02  ) 10 

003  T041302M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  T041303Y  (01  ) 

002  T041303N  (02  ) 


TCHR0077 

SCIENCE  CURRICULUM  SPECIALIST 
N04 

T041303 

CLASS 

00 

10 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


5 

4 


3 

2 


3 

2 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


003  T041303M  (M  ) 

01 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR007S 

DESCRIPTION: 

HOW  MANY  SCHOOL  HOURS  ARE  PREP  TIME  PER  WEEK? 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID : 

T056S01 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T056S01A  (01  ) 

000000 

NONE 

002  T056S01B  (02  ) 

100000 

LESS  THAN  ONE 

003  T056S01C  (03  ) 

010000 

1-2 

004  T056S01D  (04  ) 

001000 

3-4 

005  T056S01E  (05  ) 

000100 

5 

006  T056S01F  (06  ) 

000010 

MORE  THAN  5 

007  T056S01M  (M  ) 

000001 

missing 

CONDITIONING  VARIABLE  ID: 

TCHR0079 

DESCRIPTION: 

HOW  MANY  YRS  TAUGHT  SCI  IN  PUB/PRIV  SCHOOLS? 

GRADES /ASSESSMENTS : 

N04 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T06P201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060201A  (01  ) 

OOOdiJO 

NOT  TAUGHT  SCIENCE 

002  T060201B  (02  ) 

100000 

LESS  THAN  3 YEARS 

003  T060201C  (03  ) 

010000 

3-5  YEARS 

004  T060201D  (04  ) 

001000 

6-9  YEARS 

005  T060201E  (05  ) 

000100 

10-20  YEARS 

006  T060201F  (06  ) 

000010 

MORE  THAN  20  YEARS 

007  T060201M  (M  ) 

000001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHROOSO 

DESCRIPTION: 

METHODS  OF 

TCHING 

SCI?  COLLEGE  COURSE 

GRADES /ASSESSMENTS : 

N04,  NOS,  : 

SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T060301 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060301Y  (01  ) 

' 0 

YES 

002  T060301M  (M  ) 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHROOSl 

DESCRIPTION: 

METHODS  OF 

TCHING 

SCI?WRKSHP 

>1  WK 

GRADES /ASSESSMENTS : 

N04,  NOS, 

SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T060311 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060311Y  (01 

) 0 

YES 

002  T060311M  (M 

) 1 

missing 

CONDITIONING  VARIABLE  ID: 

TCHR00S2 

DESCRIPTION: 

METHODS  OF 

TCHING 

SCI?WRKSHP 

<1  WK  >1  DAY 

GRADES /ASSESSMENTS : 

N04,  NOS, 

SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T060321 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS : 

001  T060321Y  (01 

) 0 

YES 

002  T060321M  (M 

) 1 

missing 

CONDITIONING  VARIABLE  ID: 

TCHR00S3 

DESCRIPTION: 

METHODS  OF  TCHING 

SCI?WRKSHP 

<=  1 DAY 

GRADES /ASSESSMENTS : 

N04,  NOS, 

SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID : 

T060331 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060331Y  (01 

) 0 

YES 

002  T060331M  (M 

) 1 

missing 

CONDITIONING  VARIABLE  ID: 

TCHR00S4 

DESCRIPTION: 

METHODS  OF  TCHING 

SCI?0THER 

PROF.  DEV 

GRADES /ASSESSMENTS : 

N04,  NOS, 

SOS 

CONDITIONING  VAR  LABEL: 

MnPD  TD* 

T060341 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060341Y  (01 

) 0 

YES 

002  T060341M  (M 

) 1 

MISSING 

7 

6 


7 

6 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


TCHR0085 

UNIV  COURSES  IN-BIO/LIFE  SCI?  COLLEGE  COURSE 
N04,  NOS.  SOS 


T060302 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


2 


595 


BEST  COPY  AVAILABLE 


646 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060302Y  (01 

) 0 

YES 

002  T060302M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0086 

DESCRIPTION: 

UNIV  COURSES 

IN-BIO/LIFE  SCI7WRKSHP  >1  WK 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060312 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060312Y  (01 

) 0 

YES 

002  T060312M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR00S7 

DESCRIPTION: 

UNIV  COURSES 

IN-BIO/LIFE  SCI7WRKSHP  <1  WK  >1  DAY 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060322 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060322Y  (01  ] 

1 0 

YES 

002  T060322M  (M  ] 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHROOSS 

DESCRIPTION: 

UNIV  COURSES 

IN-BIO/LIFE  SCI7WRKSHP  <=  1 DAY 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060332 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060332Y  (01  ) 

0 

YES 

002  T060332M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR00S9 

DESCRIPTION: 

UNIV  COURSES 

IN-BIO/LIFE  SCI70THER  PROF.  DEV 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060342 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060342Y  (01  ) 

0 

YES 

002  T060342M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0090 

DESCRIPTION: 

UNIV  COURSES 

IN-CHEMISTRY7  COLLEGE  COURSE 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060303 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060303Y  (01  ) 

0 

YES 

002  T060303M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0091 

DESCRIPTION:  ' 

UNIV  COURSES 

IN-CHEMISTRY7WRKSHP 

>1  WK 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060313 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060313Y  (01  ) 

0 

YES 

002  T060313M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0092 

DESCRIPTION: 

UNIV  COURSES 

IN-CHEMISTRY7WRKSHP 

<1  WK  >1  DAY 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060323 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060323Y  (01  ) 

0 

YES 

002  T060323M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0093 

DESCRIPTION: 

UNIV  COURSES 

IN-CHEMISTRY7WRKSHP 

<=  1 DAY 

GRADES /ASSESSMENTS : 

N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

T060333 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS : 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T060333Y  (01  ) 

0 

YES 

002  T060333M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TCHR0094 

DESCRIPTION: 

UNIV  COURSES  : 

IN-CHEMISTRY70THER  PROF.  DEV 

ERIC 

U ' 

'i  r 

1 

2 

1 

2 

1 

2 

1 

2 

1 

2 

1 

2 

1 

2 

1 

2 

1 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


GRADES/ASSESSMENTS:  N04,  NOS,  SOS 

CONDITIONING  VAR  LABEL; 

NAEP  ID:  T060343 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060343Y  (01  ) 0 

002  T060343M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

grades/assessments : 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0095 

UNIV  COURSES  IN-PHYSICS?  COLLEGE  COURSE 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T060304 

CLASS 


001  T060304Y  (01  ) 0 

002  T060304M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0096 

UNIV  COURSES  IN-PHYSICS7WRKSHP  >1  WK 
N04,  NOS,  SOS 

T060314  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

pTace  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060314Y  (01  ) 0 

002  T060314M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0097 

UNIV  COURSES  IN-PHYSICS7WRKSHP  <1  WK  >1  DAY 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T060324 

CLASS 


001  T060324Y  (01  ) 0 

002  T060324M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR009S 

UNIV  COURSES  IN-PHYSICS7WRKSHP  <=  1 DAY 
N04,  NOS,  SOS 

T060334  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  T060334Y  (01  ) 0 

002  T060334M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0099 

UNIV  COURSES  IN-PHYSICS70THER  PROF.  DEV 
N04,  NOS,  SOS 

T060344  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  T060344Y  (01  ) 0 

002  T060344M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROlOO 

UNIV  COURSES  IN-EARTH  SCI?  COLLEGE  COURSE 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T060305 

CLASS 


001  T060305Y  (01  ) 0 

002  T060305M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHROlOl 

UNIV  COURSES  IN-EARTH  SCI?WRKSHP  >1  WK 
N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T060315 

CLASS 


001  T060315Y  (01  ) 0 

002  T060315M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0102 

UNIV  COURSES  IN-EARTH  SCI?WRKSHP  <1  WK  >1  DAY 
N04,  NOS,  SOS 

TO 6 03 2 5 TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060325Y  (01  ) 0 

002  T060325M  (M  ) 1 


YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


BEST  COPY  AVAILABLE 


597 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0103 

UNIV  COURSES  IN-EARTH  SCI7WRKSHP  <=  1 DAY 
N04,  N08,  S08 

T060335  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060335Y  (01  ) 0 

002  T060335M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0104 

UNIV  COURSES  IN-EARTH  SCI70THER  PROF.  DEV 
N04,  N08,  S08 

T060345  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060345Y  (01  ) 0 

002  T060345M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TCHR0105 

DESCRIPTION:  UNIV  COURSES-OTHER  TYPES  OF  SCI?  COLLEGE  COURSE 

GRADES /ASSESSMENTS:  N04,  N08,  S08 

CONDITIONING  VAR  LABEL: 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


NAEP  ID:  T060306 

TYPE  OF  CONTRAST:  CLASS 


001  T060306Y  (01  ) 0 

002  T060306M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0106 

UNIV  COURSES-OTHER  TYPES  OF  SCI7WRKSHP  >1  WK 
N04,  N08,  S08 

T060316  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060316Y  (01  ) 0 

002  T060316M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0107 

UNIV  COURSES-OTHR  TYPES  OF  SCI7WRKSHP  <1  WK  >1  DAY 
N04,  N08,  S08 

T060326  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060326Y  (01  ) 0 

002  T060326M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0108 

UNIV  COURSES-OTHER  TYPES  OF  SCI7WRKSHP  <=  1 DAY 
N04,  N08,  S08 

T060336  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  T060336Y  (01  ) 0 

002  T060336M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0109 

UNIV  COURSES-OTHER  TYPES  OF  SCI70THER  PROF.  DEV 
N04,  N08,  S08 

T060346  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060346Y  (01  ) 0 

002  T060346M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


TCHROllO 

UNIV  COURSES  IN-NONE  OF  ABOVE?  COLLEGE  COURSE 
N04,  N08,  S08 

T060307  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060307Y  (01  ) 0 

002  T060307M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TCHROlll 

UNIV  COURSES  IN-NONE  OF  ABOVE7WRKSHP  >1  WK 
N04,  N08,  S08 

T060317  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  T060317Y  (01  ) 0 

002  T060317M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


UNIV  COURSES  IN-NONE  OF  ABOVE7WRKSHP  <1  WK  >1  DAY 
N04,  NOS,  SOS 


T060327 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060327Y  (01  ) 0 

002  T060327M  (M  . ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
grades /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0113 

UNIV  COURSES  IN-NONE  OF  ABOVE7WRKSHP  <=  1 DAY 
N04,  NOS,  SOS 


T060337 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060337Y  (01  ) 0 

002  T060337M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0114 

UNIV  COURSES  IN-NONE  OF  AB0VE70THER  PROF.  DEV 
N04,  NOS,  SOS 


T060347 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060347Y  (01  ) 0 

002  T060347M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES / AS  SESSMENT  S : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


PAST  5 YRS,  COURSES /ACTVTS  IN-COMP  USE  TO  GET  DATA 
N04,  NOS,  SOS 


T060401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060401Y  (01  ) 0 

002  T060401M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


PAST  5 YRS,  COURSES /ACTVTS  IN-COMP  DATA  ANALYSIS 7 
N04,  NOS,  SOS 


T060402 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060402Y  (01  ) 0 

002  T060402M  (M  • ) 1 


YES 

MISSING 


2 

1 


2 

1 


2 

1 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


PAST  5 YRS,  COURSES/ACTVTS  IN-MULTIMEDIA  SCI  ED7 
N04,  nos,  SOS 


T060403 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060403Y  (01  ) 0 

002  T060403M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


PAST  5 YRS,  COURSES/ACTVTS  IN -LAB  MNGMT / S AFETY7 
N04,  nos,  SOS 


T060404 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060404Y  (01  ) 0 

002  T060404M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


PAST  5 YRS,  COURSES/ACTVTS  IN- INTEGRATED  SCI  INST7 
N04,  nos,  SOS 


T060405 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060405Y  (01  ) 0 

002  T060405M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 


YOU  BELONG  TO  1 OR  > SCI  RELATED  SCI  ORGS7 
N04,  NOS,  SOS 


2 

1 


2 

1 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VAR  LABEL: 

NAEP  ID:  T060501  • 

TYPE  OF  CONTRAST:  CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060501Y  (01  ) 00 

002  T060501N  (02  ) 10 

003  T060501M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TSUBOOOl 

HOW  OFTEN  STUDS  READ  SCI  TEXTBOOK? 
N04,  NOS,  SOS 


T060601 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060601A  (01  ) 0000 

002  T060601B  (02  ) 1000 

003  T060601C  (03  ) 0100 

004  T060601D  (04  ) 0010 

005  T060601M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES/ ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0002 

HOW  OFTEN  STUDS  READ  BOOK/MAN  ABOUT  SCI? 
N04,  NOS,  SOS 


T060602 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060602A  (01  ) 0000 

002  T060602B  (02  ) 1000 

003  T060602C  (03  ) 0100 

004  T060602D  (04  ) 0010 

005  T060602M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE /MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0003 

HOW  OFTEN  STUDS  DISCUSS  SCI  IN  THE  NEWS? 
N04,  NOS,  SOS 


T060603 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T060603A  (01  ) 0000 

002  T060603B  (02  ) 1000 

003  T060603C  (03  ) 0100 

004  T060603D  (04  ) 0010 

005  T060603M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0004 

HOW  OFTEN  STUDS  WORK  W/  OTHER  STUDS  ON  ACT/PROJCT’ 

N04,  NOS,  SOS 

T060604  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

ChASS  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  T060604A  (01  ) 0000 

002  T060604B  (02  ) 1000 

003  T060604C  (03  ) 0100 

004  T060604D  (04  ) 0010 

005  T060604M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /assessments ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


TSUB0005 

HOW  OFTEN  STUDS  GIVE  ORAL  SCI  REPORT? 

N04,  NOS,  SOS 

TO 60 605  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060605A  (01  ) 0000 

002  T060605B  (02  ) 1000 

003  T060605C  (03  ) 0100 

004  T060605D  (04  ) 0010 

005  T060605M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TSUB0006 

HOW  OFTEN  STUDS  PREPARE  A WRITTEN  SCI  REPORT? 

N04,  NOS,  SOS 

TO 60 60 6 TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060606A  (01  ) 0000 

002  T060606B  (02  ) 1000 

003  T060606C  (03  ) 0100 

004  T060606D  (04  ) 0010 

005  T060606M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID:  TSUB0007 


3 

2 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 
grades /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


HOW  OFTEN  STUDS  DO  HANDS  ON  SCI  ACTIVITIES  IN  SCI? 

N04,  NOS,  SOS 

T060607  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

P»acs  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060607A  (01  ) 0000 

002  T060607B  (02  ) 1000 

003  T060607C  (03  ) 0100 

004  T060607D  (04  ) 0010 

005  T060607M  (M  ) 0001  . 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUBOOOS 

HOW  OFTEN  STUDS  TALK  ABOUT  MEASURES/RESULTS? 
N04,  NOS,  SOS 


T06060S 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T06060SA  (01  ) 0000 

002  T06060SB  (02  ) 1000 

003  T06060SC  (03  ) 0100 

004  T06060SD  (04  ) 0010 

005  T06060SM  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0009 

HOW  OFTEN  STUDS  TAKE  SCI  TEST  OR  QUIZ? 
N04,  NOS,  SOS 


T060609 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060609A  (01  ) 0000 

002  T060609B  (02  ) 1000 

003  T060609C  (03  ) 0100 

004  T060609D  (04  ) 0010 

005  T060609M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUBOOlO 

HOW  OFTEN  STUDS  USE  LIBRARY  RESOURCES  FOR  SCI? 
N04,  NOS,  SOS 


T060610 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060610A  (01  ) 0000 

002  T060610B  (02  ) 1000 

003  T060610C  (03  ) 0100 

004  T060610D  (04  ) 0010 

005  T060610M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUBOOll 

HOW  OFTEN  STUDS  USE  COMPUTERS  FOR  SCI? 
N04,  NOS,  SOS 


T060611 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060611A  (01  ) 0000 

002  T060611B  (02  > 1000 

003  T060611C  (03  ) 0100 

004  T060611D  (04  ) 0010 

005  T060611M  (M  )-0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE /MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TSUB0012 

HOW  OFTEN  DO  YOU  TALK  TO  CLASS  ABOUT  SCI? 
N04,  NOS,  SOS 


T060701 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060701A  (01  ) 0000 

002  T060701B  (02  ) 1000 

003  T060701C  (03  ) 0100 

004  T060701D  (04  ) 0010 

005  T060701M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0013 

HOW  OFTEN  DO  YOU  DO  A SCI  DEMONSTRATION? 
N04,  NOS,  SOS 


T060702 

CLASS 


TOTAL  number  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060702A  (01  ) 0000 

002  T060702B  (02  ) 1000 

003  T060702C  (03  ) 0100 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


5 

4 


er|c 


BEST  COPY  AVAILABLE 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


004  T060702D  (04  ) 0010 

005  T060702M  (M  ) 0001 


NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0014 

HOW  OFTEN  DO  YOU  SHOW  A SCI  VIDEOTAPE /TV  PROGRAM? 
N04,  N08,  S08 


T060703 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060703A  (01  ) 0000 

002  T060703B  (02  ) 1000 

003  T060703C  (03  ) 0100 

004  T060703D  (04  ) 0010 

005  T060703M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE /MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION : 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0015 

HOW  OFTEN  DO  YOU  USE  COMPUTERS  FOR  SCI? 
N04,  N08,  S08 


T060704 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060704A  (01  ) 0000 

002  T060704B  (02  ) 1000 

003  T060704C  (03  ) 0100 

004  T060704D  (04  ) 0010 

005  T060704M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0016 

HOW  OFTEN  DO  YOU  USE  CDS  OR  LASER  DISKS  ON  SCI'^ 
N04,  N08,  S08 


T060705 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060705A  (01  ) 0000 

002  T060705B  (02  ) 1000 

003  T060705C  (03  ) 0100 

004  T060705D  (04  ) 0010 

005  T060705M  (M  ) 0001 


ALMOST  EVERY  DAY 
ONCE  OR  TWICE  A WEEK 
ONCE  OR  TWICE/MONTH 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE-  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0017 

HOW  OFTEN  YOUR  SCI  STUDS  GO  ON  SCI  FIELD  TRIPS'? 
N04,  N08,  S08 


T060801 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060801A  (01  ) 000 

002'  T060801B  (02  ) 100 

003  T06080iC  (03  ) 010 

004  T060801M  (M  ) 001 


3 OR  MORE  TIMES /YEAR 
1 OR  2 TIMES  A YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0018 

HOW  OFTEN  DO  YOU  BRING  GUEST  SPEAKER  FOR  SCI  STUDS 
N04,  N08,  S08 


T060901 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T060901A  (01  ) 000 

002  T060901B  (02  ) 100 

003  T060901C  (03  ) 010 

004  T060901M  (M  ) 001 


3 OR  MORE  TIMES /YEAR 
1 OR  2 TIMES  A YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0019 

SAVE  STUDS  SCI  WORK  IN  PORTFOLIOS  FOR  ASSESSMENT*? 
N04,  N08,  S08 


T061001 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061001Y  (01  ) 00 

002  T061001N  (02  ) 10 

003  T061001M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0020 

HOW  MUCH  EMPHASIS-KNOWING  SCI  FACTS/ TERMS'? 
N04,  N08,  S08 


T061101 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061101A  (01  ) 000 

002  T061101B  (02  ) 100 

003  T061101C  (03  ) 010 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE/NO  EMPHASIS 


5 

4 


5 

4 


5 

4 


4 

3 


4 

3 


3 

2 


4 

3 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


004  T061101M  (M  ) 001 


MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0021 

HOW  MUCH  EMPHASIS-UNDERSTANDING  KEY  SCI  CONCEPTS? 

N04,  NOS,  SOS 

T061102  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

rTaco  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061102A  (01  ) 000 

002  T061102B  (02  ) 100 

003  T061102C  (03  ) 010 

004  T061102M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0022 

HOW  MUCH  EMPHASIS -DEVELOP  SCI  PROB  SOLVING  SKILL? 

N04,  NOS,  SOS 

T061103  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  T061103A  (01  ) 000 

002  T061103B  (02  ) 100 

003  T061103C  (03  ) 010 

004  T061103M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0023 

HOW  MUCH  EMPHASIS -SCI  RELEVANCE  TO  SOCIETY /TECH? 

N04,  NOS,  SOS 

T061104  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  T061104A  (01  ) 000 

002  T061104B  (02  ) 100 

003  T061104C  (03  ) 010 

004  T061104M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0024 

HOW  MUCH  EMPHASIS-COMMUNICATE  IDEAS  IN  SCI? 

N04,  NOS,  SOS 

TO 6 1105  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

class  number  of  INDEPENDENT  CONTRASTS: 


001  T061105A  (01  ) 000 

002  T061105B  (02  ) 100 

003  T061105C  (03  ) 010 

004  T061105M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0025 

HOW  MUCH  EMPHASIS -DEVELOPING  LAB  SKILLS? 

N04,  NOS,  SOS 

T061106  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CTJVSS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061106A  (01  ) 000 

002  T061106B  (02  ) 100 

003  T061106C  (03  ) 010 

004  T061106M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0026 

HOW  MUCH  EMPHASIS -DEVELOPING  STUDS  SCI  INTEREST? 

N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T061107 

CLASS 


001  T061107A  (01  ) 000 

002  T061107B  (02  ) 100 

003  T061107C  (03  ) 010 

004  T061107M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0027 

HOW  MUCH  EMPHASIS-DEVELOPING  DATA  ANALYSIS  SKILLS 
N04,  NOS,  SOS 


T06110S 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T06110SA  (01  ) 000 

002  T06110SB  (02  ) 100 

003  T06110SC  (03  ) 010 

004  T06110SM  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE /NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID:  TSUB002S 


4 

3 


4 

3 


4 

3 


4 

3 


4 

3 


4 

3 


4 

3 


O 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


HOW  MUCH  EMPHASIS-USING  TECH  AS  SCI  TOOL’ 
N04,  NOS,  SOS 


T061109 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


001  T061109A  (01  ) 000 

002  T061109B  (02  ) 100 

003  T061109C  (03  ) 010 

004  T061109M  (M  ) 001 


HEAVY  EMPHASIS 
MODERATE  EMPHASIS 
LITTLE/NO  EMPHASIS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0029 

EVER  ASSIGN  SOLO/GROUP  SCI  PROJECTS  THAT  TAKE  >WK’ 
N04,  NOS,  SOS 


T061201 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061201Y  (01  ) 00 

002  T061201N  (02  ) 10 

003  T061201M  (M  ) 01 


YES 

NO 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  T061301A  (01  ) 00000 

002  T061301B  (02  ) 10000 

003  T061301C  (03  ) 01000 

004  T061301D  (04  ) 00100 

005  T061301E  (05  ) 00010 

006  T061301M  (M  ) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE /GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


TSUB0030 

HOW  OFTEN  USE  MULT  CHOICE  TESTS  TO  ACCESS’ 
N04,  NOS,  SOS 

T061301 
CLASS 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0031 

HOW  OFTEN  USE  SHOR/LONG  WRITTEN  RESPONSE  TO  ACCESS 
N04,  NOS,  SOS 


T061302 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061302A  (01  ) 00000 

002  T061302B  (02  ) 10000 

003  T061302C  (03  ) 01000 

004  T061302D  (04  ) 00100 

005  T061302E  (05  ) 00010 

006  T061302M  (M  ) 00001 


ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


USE  SOLO  PROJECTS  TO  ACCESS? 
SOS 


CONDITIONING 

VARIABLE  ID: 

TSUB0032 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 

N04,  NOS, 

CONDITIONING 

VAR  LABEL: 

NAEP  ID: 

T061303 

TYPE  OF  CONTRAST: 

CLASS 

001  T061303A 

(01  ) 

1 00000 

002  T061303B 

(02  ) 

' 10000 

003  T061303C 

(03  ) 

' 01000 

004  T061303D 

(04  ) 

' 00100 

005  T061303E 

(05  ) 

00010 

006  T061303M 

(M  ) 

00001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0033 

HOW  OFTEN  USE  GROUP  PROJECTS  TO  ACCESS? 
N04,  NOS,  SOS 


T061304 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061304A  (01  ) 00000 

002  T061304B  (02  ) 10000 

003  T061304C  (03  ) 01000 

004  T061304D  (04  ) 00100 

005  T061304E  (05  ) 00010 

006  T061304M  (M  ) 00001 


ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0034 

HOW  OFTEN  USE  WORK  PORTFOLIOS  TO  ACCESS? 

N04,  NOS,  SOS 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


T061305 

CLASS 


001  T061305A  (01  ) 00000 

002  T061305B  (02  ) 10000 


ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


003 

T061305C 

(03 

) 01000 

004 

T061305D 

(04 

) 00100 

005 

T061305E 

(05 

) 00010 

006 

T061305M 

(M 

) 00001 

ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0035 

HOW  OFTEN  USE  IN  CLASS  ESSAYS  TO  ACCESS? 

N04,  NOS,  SOS 

TO 613 06  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T061306A  (01  ) 00000 

002  T061306B  (02  ) 10000 

003  T061306C  (03  ) 01000 

004  T061306D  (04  ) 00100 

005  T061306E  (05  ) 00010 

006  T061306M  (M  ) 00001 


ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


6 

5 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

grades/assessments : 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST : 


001  T061307A  (01  ) 00000- 

002  T061307B  (02  ) 10000 

003  T061307C  (03  ) 01000 

004  T061307D  (04  ) 00100 

005  T061307E  (05  ) 00010 

006  T061307M  (M  ) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


TSUB0036 

HOW  OFTEN  USE  SELF/PEER  EVAL  TO  ACCESS? 
N04,  NOS,  SOS 

T061307 
CLASS 


6 

5 


CONDITIONING  VARIABLE  ID:  TSUB0037 

DESCRIPTION:  HOW  OFTEN  USE  LAB  NOTEBOOKS/ JOURNALS  TO  ACCESS? 


GRADES /ASSESSMENTS : 

N04,  NOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

T06130S 

CLASS 

001  T06130SA 

(01 

) 00000 

002  T06130SB 

(02 

) 10000 

003  T06130SC 

(03 

) 01000 

004  T06130SD 

(04 

) 00100  ■ 

005  T06130SE 

(05 

) 00010 

006  T06130SM 

(M 

) 00001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  6 

NUMBER  OF  INDEPENDENT  CONTRASTS:  5 

ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB003S 

HOW  OFTEN  USE  HOMEWORK  TO  ACCESS? 

N04,  NOS,  SOS 

T061309  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

T061309A 

(01 

) 00000 

002 

T061309B 

(02 

) 10000 

003 

T061309C 

(03 

) 01000 

004 

T061309D 

(04 

) 00100 

005 

T061309E 

(05 

) 00010 

006 

T061309M 

(M 

) 00001 

ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0039 

HOW  OFTEN  USE  HANDS  ON  ACTIVITIES  TO  ACCESS? 
N04,  NOS,  SOS 


T061310 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

T061310A 

(01 

) 00000 

002 

T061310B 

(02 

) 10000 

003 

T061310C 

(03 

) 01000 

004 

T061310D 

(04 

) 00100 

005 

T061310E 

(05 

) 00010 

006 

T061310M 

(M 

) 00001 

ONCE  OR  TWICE  WEEK 
ONCE  OR  TWICE  MONTH 
ONCE/GRADING  PERIOD 
ONCE  OR  TWICE/YEAR 
NEVER  OR  HARDLY  EVER 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0040 

PROPORTION  OF  EVAL  IN  SCI  BASED  ON  HANDS  ON  ACTVS? 
N04,  NOS,  SOS 


T061401 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

T061401A 

(01 

) 0000 

002 

T061401B 

(02 

) 1000 

003 

T061401C 

(03 

) 0100 

004 

T061401D 

(04 

) 0010 

005 

T061401M 

(M 

) 0001 

MOST  OR  ALL  OF  GRADE 
ABOUT  HALF  OF  GRADE 
VERY  LITTLE  OF  GRADE 
NONE  OF  GRADE 
MISSING 


6 

5 


6 

5 


5 

4 


BEST  COPY  AVAILABLE 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  ID: 

TSUB0041 

DESCRIPTION: 

BEST  DESCRIPTION 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061501 

TYPE  OF  CONTRAST: 

CLASS 

001  T061501A  (01 

) oooobo 

002  T061501B  (02 

) 100000 

003  T061501C  (03 

) 010000 

004  T061501D  (04 

) 001000 

005  T061501E  (05 

) 000100 

006  T061501F  (06 

) 000010 

007  T061501M  (M 

) 000001 

CONDITIONING  VARIABLE  ID: 

TSUB0042 

DESCRIPTION: 

USE  COMPUTERS 

FOR 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061601 

TYPE  OF  CONTRAST: 

CLASS 

001  T061601Y  (01 

) 0 

002  T061601M  (M 

) 1 

CONDITIONING  VARIABLE  ID: 

TSUB0043 

DESCRIPTION: 

USE  COMPUTERS 

FOR 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061611 

TYPE  OF  CONTRAST: 

CLASS 

001  T061611Y  (01  ] 

1 0 

002  T061611M  (M  ] 

1 1 

CONDITIONING  VARIABLE  ID: 

TSUB0044 

DESCRIPTION: 

USE  COMPUTERS 

FOR 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061621 

TYPE  OF  CONTRAST: 

CLASS 

001  T061621Y  (01  ) 

0 

002  T061621M  (M  ) 

1 

CONDITIONING  VARIABLE  ID: 

TSUB0045 

DESCRIPTION: 

USE  COMPUTERS 

FOR 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061631 

TYPE  OF  CONTRAST: 

CLASS 

001  T061631Y  (01  ) 

0 

002  T061631M  (M  ) 

1 

CONDITIONING  VARIABLE  ID: 

TSUB0046 

DESCRIPTION: 

USE  COMPUTERS 

FOR 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061641 

TYPE  OF  CONTRAST: 

CLASS 

001  T061641Y  (01  ) 

0 

002  T061641M  (M  ) 

1 

CONDITIONING  VARIABLE  ID: 

TSUB0047 

DESCRIPTION: 

USE  COMPUTERS 

FOR  , 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID: 

T061651 

TYPE  OF  CONTRAST: 

CLASS 

001  T061651Y  (01  ) 

0 

002  T061651M  (M  ) 

1 

CONDITIONING  VARIABLE  ID: 

TSUB004S 

DESCRIPTION: 

STUDS  assigned 

TO  ( 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS,  SOS 

NAEP  ID : 

T061701 

TYPE  OF  CONTRAST: 

CLASS 

001  T061701A  (01  ) 

00000 

002  T061701B  (02  ) 

10000 

003  T061701C  (03  ) 

01000 

004  T061701D  (04  ) 

00100 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE  AVAILABLE 
1 IN  CLASSROOM 
2-3  IN  CLASSROOM 
4 IN  CLASSROOM 
IN  LAB  BUT  DIFFICULT 
IN  LAB  AND  EASY 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


PLAYING  SCI 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NOT  assigned  by  ABIL 
ASSIGNED  BY  MATN  ABL 
ASSIGNED  -BY  READING 
ASSIGNED  BY  SCIENCE 


ERIC 


6 


j /. 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


005  T061701E  (05  ) 00010 

006  T061701M  (M  ) 00001 


ASSIGNED  BY  GENERAL 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0049 

IF  assigned  by  ability  which  best  describes  LEVEL? 

N04,  NOS,  SOS 

T061S01  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


6 

5 


001 

T061S01A 

(01 

) 00000 

002 

T061S01B 

(02 

) 10000 

003 

T061S01C 

(03 

) 01000 

004 

T061S01D 

(04 

) 00100 

005 

T061S01E 

(05 

) 00010 

006 

T061S01M 

(M 

) 00001 

NOT  assigned  by  ABIL 
MOSTLY  HIGH  ABILITY 
MOSTLY  AVERAGE  ABIL 
MOSTLY  LOW  ABILITY 
MOSTLY  MIXED  ABILITY 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0050 

COMPOSITION  OF  CLASS  ACCORDING  TO  GENDER? 

N04,  NOS,  SOS 

T061901  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


4 

3 


001 

T061901A 

(01 

) 000 

002 

T061901B 

(02 

) 100 

003 

T061901C 

(03 

) 010 

004 

T061901M 

(M 

) 001 

MIXED 
ALL  MALE 
ALL  FEMALE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0051 

HOW  MUCH  TIME  CLASS  SPEND  ON  LIFE  SCIENCE? 

N04,  NOS,  SOS 

T062001  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001 

T062001A 

(01 

) 0000 

002 

T062001B 

(02 

) 1000 

003 

T062001C 

(03 

) 0100 

004 

T062001D 

(04 

) 0010 

005 

T062001M 

(M 

) 0001 

A LOT 

SOME 

LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0052 

HOW  MUCH  TIME  CLASS  SPEND  ON  EARTH  SCIENCE? 

N04,  NOS,  SOS 

T062002  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


5 

4 


001 

T062002A 

(01 

) 0000 

002 

T062002B 

(02 

) 1000 

003 

T062002C 

(03 

) 0100 

004 

T062002D 

(04 

) 0010 

005 

T062002M 

(M 

) 0001 

A LOT 

SOME 

LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0053 

HOW  MUCH  TIME  CLASS  SPEND  ON  PHYSICAL  SCIENCE? 

N04,  NOS,  SOS 

T062003  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001 

T062003A 

(01 

) 0000 

002 

T062003B 

(02 

) 1000 

003 

T062003C 

(03 

) 0100 

004 

T062003D 

(04 

) 0010 

005 

T062003M 

(M 

) 0001 

A LOT 

SOME 

LITTLE 

NONE 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

TSUB0054 

WHICH  BEST  DESCRIBES  SPACE  WHERE  CLASS  TAUGHT? 

GRADES /ASSESSMENTS : 

N04,  NOS, 

SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T062101 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  T062101A  (01  1 

f 0000 

CLASS  ONLY 

002  T062101B  (02  1 

f 1000 

CLASS  W/  H20  ACCESS 

003  T062101C  (03  1 

f 0100 

CLASS  W/  LAB  ACCESS 

004  T062101D  (04  1 

1 0010 

LAB  W/  H20  ACCESS 

005  T062101M  (M  1 

1 0001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

TSUB0055 
DO  STUDS 

PRODUCE 

NOTEBOOKS/REPORTS  OF  LAB  WORK? 

GRADES/ASSESSMENTS : 

N04,  NOS, 

SOS 

CONDITIONING  VAR  LABEL; 
NAEP  ID; 

T062201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

5 

4 


2 

1 


best  COPY  AVAILABLE 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  T062201Y  (01  ) 0 

002  T062201M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0056 

DO  STUDS  PRODUCE  REPORTS  OF  EXTENDED  SCI  PROJECTS? 

N04,  NOS,  SOS 

T062202  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T062202Y  (01  ) 0 

002  T062202M  (M  ) 1 


YES 

MISSING 


conditioning  variable  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0057 

DO  STUDS  PRODUCE  REPORTS  ON  SPECIFIC  TOPIC/ ISSUE? 

N04,  NOS,  SOS 

T062203  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T062203Y  (01  ) 0 

002  T062203M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST; 


TSUB005S 

DO  STUDS  PRODUCE  REPORTS /RECORDS  OF  FIELD  TRIPS? 

N04,  NOS,  SOS 

T062204  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T062204Y  (01  ) 0 

002  T062204M  (M  } 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST: 


TSUB0059 

DO  STUDS  PRODUCE  JOURNALS/DIARIES/LOGS  OF  IDEAS? 

N04,  NOS,  SOS 

TO 622 05  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  . 


001  T062205Y  (01  ) 0 

002  T062205M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0060 

DO  STUDS  PRODUCE  PHOTO  RECORDS  OF  PROJECTS? 

N04,  NOS,  SOS 

T062206  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T062206Y  (01  ) 0 YES 

002  T062206M  (M  ' ) 1 MISSING 

CONDITIONING  VARIABLE  ID:  TSUB0061 

DESCRIPTION:  DO  STUDS  PRODUCE  AUDIO/VIDEOTAPE  RECORDS  OF  ACTVS? 

GRADES /ASSESSMENTS:  N04 , NOS,  SOS 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  T' 

TYPE  OF  CONTRAST:  C 

001  T062207Y  (01  ) 0 

002  T062207M  (M  ) 1 


062207 

LASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0062 

DO  STUDS  PRODUCE  REPORTS  OF  PERSONAL  INTERVIEWS? 

N04,  NOS,  SOS 

TO 622 OS  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T06220SY  (01  ) 0 

002  T06220SM  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TSUB0063 

DO  STUDS  PRODUCE  3D  SCI  MODELS? 

N04,  NOS,  SOS 

T062209  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  T062209Y  (01  ) 0 

002  T062209M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID:  TSUB0064 

DESCRIPTION:  DO  STUDS  PRODUCE  COMP  GENERATED  MULTMEDIA  PROJECTS 

GRADES /ASSESSMENTS : N04,  NOS,  SOS 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


2 

1 


O 08 

ERIC 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VAR  LABEL: 
NAEP  ID: 

T062210 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

2 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

1 

001  T062210Y  (01 

) 0 

YES 

002  T062210M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

TSUB0065 

DESCRIPTION: 

TIME  PER 

WEEK  EXPECT 

STUD  TO  SPEND  ON  HOMEWORK? 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

, SOS 

NAEP  ID: 

T062301 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

6 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

5 

001  T062301A  (01 

) 00000 

NONE 

002  T062301B  (02 

) 10000 

1/2  HOUR 

003  T062301C  (03 

) 01000 

1 HOUR 

004  T062301D  (04 

) 00100 

2 HOURS 

005  T062301E  (05 

) 00010 

MORE  THAN  2 HOURS 

006  T062301M  (M 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

TSUB0066 

DESCRIPTION: 

CLASS  PERIOD  AND  # 

OF 

STUDS 

IN  CLASS 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  NOS, 

SOS 

NAEP  ID: 

T062401 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

6 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

5 

001  T062401A  (01  ] 

I 00000 

1-20  STUDENTS 

002  T062401B  (02  ] 

1 10000 

21025  STUDENTS 

003  T062401C  (03  ] 

1 01000 

26-30  STUDENTS 

004  T062401D  (04  ] 

1 00100 

31-35  STUDENTS 

005  T062401E  (05  ] 

1 00010 

36  OR  MORE  STUDENTS 

006  T062401M  (M  ] 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL00S7 

DESCRIPTION: 

BEST  DESCRIBES 

HOW 

STH  GRADES  ARE  ORGANIZED? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

C034201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

4 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

3 

001  C034201A  (01  ) 

< 000 

SELF-CONTAINED 

002  C034201B  (02  ) 

1 100 

SEMI-DEPARTMENTALIZE 

003  C034201C  (03  ) 

1 010 

DEPARTMENTALIZED 

004  C034201M  (M  ) 

1 001 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHLOOSS 

DESCRIPTION: 

ARE  STH -GRADERS 

ASSIGNED 

TO 

MATH  BY  ABILITY? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID : 

C034402 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  ‘ 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

001  C034402Y  (01  ) 

1 00 

YES 

002  C034402N  (02  ] 

1 10 

NO 

003  C034402M  (M  ] 

1 01 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL00S9 

DESCRIPTION: 

ARE  STH -GRADERS 

ASSIGNED 

TO 

SCIENCE  BY  ABILITY? 

GRADES /ASSESSMENTS : 

NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

C034403 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

001  C034403Y  (01  ) 

I 00 

YES 

002  C034403N  (02  ) 

1 10 

NO 

003  C034403M  (M  ] 

1 01 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0090 

DESCRIPTION: 

ARE  STH-GRADERS 

ASSIGNED 

TO 

ENGLISH  BY  ABILITY? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

C034401 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

001  C034401Y  (01  ) 

1 00 

YES 

002  C034401N  (02  ) 

1 10 

NO 

003  C034401M  (M  ] 

1 01 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0091 

DESCRIPTION: 

ARE  STH-GRADERS 

ASSIGNED 

TO 

ARTS  BY  ABILITY? 

GRADES/ASSESSMENTS : 

NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

C034406 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

3 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

609 

660 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  C034406Y  (01 

) 00 

002  C034406N  (02 

) 10 

003  C034406M  (M 

) 01 

CONDITIONING  VARIABLE  ID: 

SCHL0092 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08,  S08 

NAEP  ID: 

C034510 

TYPE  OF  CONTRAST: 

CLASS 

001  C034510A  (01 

) 00000 

002  C034510B  (02 

) 10000 

003  C034510C  (03 

) 01000 

004  C034510D  (04 

) 00100 

005  C034510E  (05 

) 00010 

006  C034510M  (M 

) 00001 

CONDITIONING  VARIABLE  ID: 

SCHL0093 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08,  S08 

NAEP  ID: 

C034511 

TYPE  OF  CONTRAST: 

CLASS 

001  C034511A  (01  ] 

1 00000 

002  C034511B  (02  ] 

1 10000 

003  C034511C  (03  ] 

1 01000 

004  C034511D  (04  ) 

i 00100 

005  C034511E  (05  ) 

i 00010 

006  C034511M  (M  ) 

1 00001 

CONDITIONING  VARIABLE  ID: 

SCHL0094 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08,  S08 

NAEP  ID: 

C034512 

TYPE  OF  CONTRAST: 

CLASS 

001  C034512A  (01  ) 

00000 

002  C034512B  (02  ) 

10000 

003  C034512C  (03  ) 

01000 

004  C034512D  (04  ) 

00100 

005  C034512E  (05  ) 

00010 

006  C034512M  (M  ) 

00001 

CONDITIONING  VARIABLE  ID: 

SCHL0095 

DESCRIPTION: 

HOW  OFTEN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N08,  S08 

NAEP  ID: 

C034513 

TYPE  OF  CONTRAST: 

CLASS 

001  C034513A  (01  ) 

00000 

002  C034513B  (02  ) 

10000 

003  C034513C  (03  ) 

01000 

004  C034513D  (04  ) 

00100 

005  C034513E  (05  ) 

00010 

006  C034513M  (M  ) 

00001 

YES 

NO 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE /WEEK 
LESS  THAN  ONCE /WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE /WEEK 
LESS  THAN  ONCE /WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE /WEEK 
LESS  THAN  ONCE /WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 Times  a week 
ONCE  OR  TWICE /WEEK 
LESS  THAN  ONCE /WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0096 

HOW  OFTEN  8TH  GRDS  RECEIVE  ARTS  INSTRUCTION? 
N08,  S08 


001  C034514A  (01 

002  C034514B  (02 

003  C034514C  (03 

004  C034514D  (04 

005  C034514E  (05 

006  C034514M  (M 


C034514 

CLASS 

) 00000 
) 10000 
) 01000 
) 00100 
) 00010 
) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

EVERY  DAY 

3 OR  4 TIMES  A WEEK 
ONCE  OR  TWICE /WEEK 
LESS  THAN  ONCE /WEEK 
SUBJECT  NOT  TAUGHT 
MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0097 

DESCRIPTION: 

GRADES /ASSESSMENTS : 

CONDITIONING  VAR  LABEL: 

NAEP  ID:  C031611 

TYPE  OF  CONTRAST:  CLASS 


HAS  ENGLISH  BEEN  IDENTIFIED  AS  A PRIORITY? 
N08,  S08 


001  C031611Y  (01 

002  C031611N  (02 

003  C031611M  (M 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 


) 00 
) 10 
) 01 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

NO 

MISSING 


SCHL0098 

SCHOOL  OFFER  8TH  GR  STUDS  ALGEBRA  FOR  HS  CREDIT? 


ERIC 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C034601  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C034601Y  (01 

002  C034601N  (02 

003  C034601M  (M 

) 00  YES 

) 10  NO 

) 01  MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0099 

SCHOOL  W/  SPECIAL  FOCUS  ON  ENGLISH? 
NOS,  SOS,  N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037203  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037203Y  (01  ! 

002  C037203M  (M  ! 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHLOlOO 

SCHOOL  FOLLOW  DISTRICT/STATE  ENGLISH  CURRICULUM? 
NOS,  SOS,  N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C037306  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C037306Y  (01  1 

002  C037306M  (M  ; 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHLOlOl 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  MATH? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C039401Y  (01  . ] 

002  C039401M  (M  ] 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0102 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  SCIENCE? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039402  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C039402Y  (01  ) 

002  C039402M  (M  ) 

1 0 YES 

' 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0103 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  READING? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039403  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C039403Y  (01  ) 

002  C039403M  (M  ) 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0104 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  ARTS? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039404  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C039404Y  (01  ) 

002  C039404M  (M  ) 

1 0 YES  : 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0105 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  OTHER? 

NOS,  SOS  ’ ' 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039405  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C039405Y  (01  ) 

002  C039405M  (M  ) 

1 0 YES 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

grades/assessments : 

CONDITIONING  VAR  LABEL: 

SCHL0106 

SCHOOL  SPONSER  STH  GRDS  FIELD  TRIP  FOR  NONE  ABOVE? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039406  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C039406Y  (01  ) 

1 0 YES 

ti02 


BEST  COPY  AVAILABLE 


Table  C*5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


002  C039406M  (M 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0107 

8TH  GRADERS  IN  EXTRACURR  ACTS  FOR  MATH? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039501  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C039501Y  (01 

002  C039501M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHLOIOS 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  SCIENCE? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039502  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C039502Y  (01 

002  C039502M  (M 

) 0 yes 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0109 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  ENG/LANG  ARTS? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039503  TOTAL  NUMBER  OF  SPECIFIED  CONTI^STS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C039503Y  (01  ) 

002  C039503M  (M  ) 

' 0 yes 

' 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHLOllO 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  ARTS? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039504  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C039504Y  (01  ) 

002  C039504M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHLOlll 

STH  GRADERS  IN  EXTRACURR  ACTS  FOR  NONE  OF  ABOVE? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039505  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C039505Y  (01  ) 

002  C039505M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0112 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  MATH? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C039601  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C039601Y  (01  ) 

002  C039601M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

SCHL0113 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  SCIENCE? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST; 

C039602  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C039602Y  (01  ) 

002  C039602M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 

SCHL0114 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  ENG/LANG  ARTS? 
NOS,  SOS 

NAEP  ID: 

TYPE  OF  CONTRAST; 

C039603  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C039603Y  (01  ) 

002  C039603M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0115 

STH  GRADERS  IN  SUMMER  PROGRAMS  IN  ARTS? 
NOS,  SOS 

NAEP  ID: 

C039604  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 


CLASS 


NUMBER  OF  INDEPENDENT  CONTRASTS:  1 


001  C039604Y  (01  ) 0 

002  C039604M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0116 

8TH  GRADERS  IN  SUMMER  PROGRAMS  IN  NONE  OF  ABOVE? 

NOS,  SOS 

C039605  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C039605Y  (01  ) 0 

002  C039605M  (M  ) 1 


YES 

MISSING 


2 

1 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


001  C041901A  (01  ) 00000 

002  C041901B  (02  ) 10000 

003  C041901C  (03  ) 01000 

004  C041901D  (04  ) 00100 

005  C041901E  (05  ) 00010 

006  C041901M  (M  ) 00001 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

0% 

1-2% 

3-5% 

6-10% 

MORE  THAN  10% 

MISSING 


SCHL0117 

WHAT  % OF  STH  GRDS  HELD  BACK/REPEAT  8TH  GRADE? 
NOS,  SOS 

C041901 
CLASS 


6 

5 


CONDITIONING  VARIABLE  ID:  TCHR0121 

DESCRIPTION:  COUNTING  THIS  YR , HOW  MANY  YRS  TOTAL  TAUGHT  SCI? 


GRADES /ASSESSMENTS : 

NOS,  SOS 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

T062501 

CLASS 

001  T062501A 

(01 

) 00000 

002  T062501B 

(02 

) 10000 

003  T062501C 

(03 

) 01000 

004  T062501D 

(04 

) 00100 

005  T062501E 

(05 

) 00010 

006  T062501M 

(M 

) 00001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  6 

NUMBER  OF  INDEPENDENT  CONTRASTS:  5 

2 YEARS  OR  LESS 
3-5  YEARS 
6-10  YEARS 
11-24  YEARS 
25  YEARS  OR  MORE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0122 

LAST  YR,  TIME  IN  PRO  WORKSHOPS/ SEMS  IN  SCI? 

NOS,  SOS 

T062601  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


6 

5 


001 

T062601A 

(01 

) 00000 

002 

T062601B 

(02 

) 10000 

003 

T062601C 

(03 

) 01000 

004 

T062601D 

(04 

) 00100 

005 

T062601E 

(05 

) 00010 

006 

T062601M 

(M 

) 00001 

NONE 

LESS  THAN  6 HOURS 
6-15  HOURS 
16-35  HOURS 
MORE  THAN  35  HOURS 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0123 

LAST  2 YRS,  # OF  UNIV  COURSES  IN  SCI/SCI  ED? 

NOS,  SOS 

T062701  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


6 

5 


001 

T062701A 

(01 

) 00000 

002 

T062701B 

(02 

) 10000 

003 

T062701C 

(03 

) 01000 

004 

T062701D 

(04 

) 00100 

005 

T062701E 

(05 

) 00010 

006 

T062701M 

(M 

) 00001 

NONE 

ONE 

TWO 

THREE 

FOUR  OR  MORE 
MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


TCHR0124 

CURRICULUM  SPECIALIST  TO  HELP/ADVISE  IN  SCI? 

NOS,  SOS 

TO 62 801  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


3 

2 


001 

T062801Y 

(01 

) 00 

YES 

002 

T062801N 

(02 

) 10 

NO 

003 

T062801M 

(M 

) 01 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


SCHL0118 

ARE  12TH-GRADERS  ASSIGNED  TO  MATH  BY  ABILITY? 

N12 

C035002  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


3 


613 


BEST  COPY  AVAILABLE 


W V. 


664 


Table  C-5  (continued) 


Conditioning  Variables  Specific  to  the  1996  Science  Assessment 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C035002Y  (01 

) 00 

YES 

002  C035002N  (02 

) 10 

NO 

003  C035002M  (M 

) 01 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0119 

DESCRIPTION: 

ARE  12TH-GRADERS 

ASSIGNED 

TO 

SCIENCE  BY  ABILITY? 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C035003 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS : 

001  C035003Y  (01 

1 00 

YES 

002  C035003N  (02 

» 10 

NO 

003  C035003M  (M 

» 01 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0120 

DESCRIPTION: 

ARE  12TH-GRADERS 

ASSIGNED 

TO 

ENGLISH  BY  ABILITY? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL 

N12 

NAEP  ID: 

C035006 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C035006Y  (01 

] 

1 00 

YES 

002  C035006N  (02 

] 

1 10 

NO 

003  C035006M  (M 

] 

1 01 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0121 

DESCRIPTION: 

ARE  12TH-GRADERS 

ASSIGNED 

TO 

ARTS  BY  ABILITY? 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL 

N12 

NAEP  ID: 

C035007 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C035007Y  (01 

) 

00 

YES 

002  C035007N  (02 

) 

10 

NO 

003  C035007M  (M 

) 

01 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0122 

DESCRIPTION: 

FROM  9TH  ON  HOW  MANY  YRS  REQUIRED  FOR  MATH? 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL 

N12 

NAEP  ID: 

C040201 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040201A  (01 

) 

0000000 

NONE 

002  C040201B  (02 

) 

1000000 

ONE  HALF  YEAR 

003  C040201C  (03 

) 

0100000 

ONE  YEAR 

004  C040201D  (04 

) 

0010000 

TWO  YEARS 

005  C040201E  (05 

) 

0001000 

THREE  YEARS 

006  C040201F  (06 

) 

0000100 

FOUR  YEARS 

007  C040201G  (07 

) 

0000010 

MORE  THAN  FOUR  YEARS 

008  C040201M  (M 

) 

0000001 

MISSING 

CONDITIONING  VARIABLE 

ID: 

SCHL0123 

DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


FROM  9TH  ON  HOW  MANY  YRS  REQUIRED  FOR  SCIENCE? 
N12 


C040202 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 


3 

2 


3 

2 


3 

2 


8 

7 


8 

7 


001  C040202A 

(01  ) 

1 0000000 

NONE 

002  C040202B 

(02  ) 

1 1000000 

ONE  HALF  YEAR 

003  C040202C 

(03  ) 

1 0100000 

ONE  YEAR 

004  C040202D 

(04  ) 

1 0010000 

TWO  YEARS 

005  C040202E 

(05  1 

1 0001000 

THREE  YEARS 

006  C040202F 

(06  1 

1 0000100 

FOUR  YEARS 

007  C040202G 

(07  ) 

1 0000010 

MORE  THAN  FOUR  YEARS 

008  C040202M 

(M  ) 

1 0000001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

SCHL0124 
FROM  9TH 
N12 

C040203 

CLASS 

ON  HOW  MANY  YRS  REQUIRED  FOR  ENG/ LIT? 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040203A 

(01  ; 

I 0000000 

NONE 

002  C040203B 

(02  ; 

1 1000000 

ONE  HALF  YEAR 

003  C040203C 

(03  ; 

1 0100000 

ONE ‘year 

004  C040203D 

(04  ; 

» 0010000 

TWO  YEARS 

005  C040203E 

(05  ; 

» 0001000 

THREE  YEARS 

006  C040203F 

(06  ; 

1 0000100 

FOUR  YEARS 

007  C040203G 

(07  ; 

» 0000010 

MORE  THAN  FOUR  YEARS 

008  C040203M 

(M  ; 

1 0000001 

MISSING 

ERIC 


6G5 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


CONDITIONING  VARIABLE  : 

[D: 

SCHL0125 

DESCRIPTION: 

FROM  9TH 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040204 

TYPE  OF  CONTRAST: 

CLASS 

001  C040204A  (01 

) 

0000000 

002  C040204B  (02 

) 

1000000 

003  C040204C  (03 

) 

0100000 

004  C040204D  (04 

) 

0010000 

005  C040204E  (05 

) 

0001000 

006  C040204F  (06 

) 

0000100 

007  C040204G  (07 

) 

0000010 

008  C040204M  (M 

) 

0000001 

CONDITIONING  VARIABLE  : 

ID: 

SCHL0126 

DESCRIPTION: 

COURSES  1 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040301 

TYPE  OF  CONTRAST: 

CLASS 

001  C040301Y  (01 

) 

0 

002  C040301M  (M 

) 

1 

CONDITIONING  VARIABLE 

ID: 

SCHL0127 

DESCRIPTION: 

COURSES  1 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040302 

TYPE  OF  CONTRAST: 

CLASS 

001  C040302Y  (01 

) 

0 

002  C040302M  (M 

) 

1 

CONDITIONING  VARIABLE 

ID: 

SCHL0128 

DESCRIPTION: 

COURSES  1 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040303 

TYPE  OF  CONTRAST: 

CLASS 

001  C040303Y  (01 

) 

1 0 

002  C040303M  (M 

) 

1 1 

CONDITIONING  VARIABLE 

ID: 

SCHL0129 

DESCRIPTION: 

NO  ADVANC 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040304 

TYPE  OF  CONTRAST: 

CLASS 

001  C040304Y  (01 

1 0 

002  C040304M  (M 

1 1 

CONDITIONING  VARIABLE 

ID: 

SCHL0130 

DESCRIPTION: 

COURSES  ] 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040305 

TYPE  OF  CONTRAST: 

CLASS 

001  C040305Y  (01 

) 0 

002  C040305M  (M 

) 1 

CONDITIONING  VARIABLE 

ID: 

SCHL0131 

DESCRIPTION: 

COURSES  : 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040306 

TYPE  OF  CONTRAST: 

CLASS 

001  C040306Y  (01 

) 0 

002  C040306M  (M 

) 1 

CONDITIONING  VARIABLE 

ID: 

SCHL0132 

DESCRIPTION: 

COURSES 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

ONE  HALF  YEAR 
ONE  YEAR 
TWO  YEARS 
THREE  YEARS 
FOUR  YEARS 

MORE  THAN  FOUR  YEARS 
MISSING 


OR  > SEMESTERS  TAUGHT  IN  ADVANCED  BIO? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF.  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


OR  > SEMESTERS  TAUGHT  IN  ADVANCED  CHEM? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


OR  > SEMESTERS  TAUGHT  IN  ADV  PHYSICS? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


OR  > SEMESTERS  TAUGHT  IN  COMP  SCI? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


YES 

MISSING 


OR  > SEMESTERS  TAUGHT  IN  CALCULUS? 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


1 OR  > SEMESTERS  TAUGHT  IN  TRIGONOMETRY? 


GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

001  C040307Y  (01 

002  C040307M  (M 


N12 

C040307 

CLASS 

0 

1 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


CONDITIONING  VARIABLE  ID:  SCHL0133 


fci66 


615 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  PRECALCULUS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040308  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040308Y  (01 

002  C040308M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0134 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  PROB/STAT? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040309  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040309Y  (01 

002  C040309M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0135 

COURSES  1 OR  > SEMESTERS  TAUGHT  IN  UNI/INTEG  MATH? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040310  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040310Y  (01  1 

002  C040310M  (M  1 

1 0 yes 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0136 

NO  ADVANCED  MATH  COURSES  TAUGHT? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040311  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040311Y  (01  1 

002  C040311M  (M  ] 

' 0 yes 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0137 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  MATH? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040401  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040401Y  (01  ) 

002  C040401N  (02  ) 

003  C040401M  (M  ) 

' 00  YES 

' 10  NO 

' 01  MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0138 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  SCIENCE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040402  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040402Y  (01  ) 

002  C040402N  (02  ) 

003  C040402M  (M  ) 

00  yes 

10  NO 

01  MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0139 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  ENG/LANG  ARTS 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040403  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040403Y  (01  ) 

002  C040403N  (02  ) 

003  C040403M  (M  ) 

00  YES 

10  NO 

01  MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0140 

STUDS  REQUIRED  TO  PASS  STATE  TEST  IN  FINE/PERF  ART 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040404  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040404Y  (01  ) 

002  C040404N  (02  ) 

003  C040404M  (M  ) 

00  YES 

10  NO 

01  MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0141 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  FOR  MATH? 
N12 

KIC 

Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


NAEP  ID: 

C040501 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040501Y  (01  ■ ■) 

1 0 ■ 

YES 

002  C040501M  (M  ) 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0142 

DESCRIPTION: 

SCHOOL  SPONSER 

12TH 

GR 

FIELD 

TRIPS  FOR  SCIENCE? 

grades /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040502 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040502Y  (01  1 

1 0 

YES 

002  C040502M  (M  1 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0143 

DESCRIPTION: 

SCHOOL  SPONSER 

12TH 

GR 

FIELD 

TRIPS  IN  ENG/LANG  ART 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040503 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040503Y  (01  ] 

1 0 

YES 

002  C040503M  (M  ] 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0144 

DESCRIPTION: 

SCHOOL  SPONSER 

12TH 

GR 

FIELD 

TRIPS  IN  ARTS? 

GRADES/ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040504 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

2 

1 


2 

1 


2 

1 


2 

1 


001  C040504Y  (01  ) 0 

002  C040504M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0145 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  IN  OTHER? 

N12 

C040505  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C040505Y  (01  ) 0 

002  C040505M  (M  ) 1 


YES 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


SCHL0146 

SCHOOL  SPONSER  12TH  GR  FIELD  TRIPS  IN  NONE  ABOVE? 

N12 

C040506  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


2 

1 


2 

1 


OOr  C040506Y  (01  ) 

0 

YES 

002  C040506M  (M  ) 

1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0147 

DESCRIPTION: 

12TH  GRADERS 

IN 

EXTRACURR 

ACTS 

IN  MATH? 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040601 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040601Y  (01  ) 

1 0 

YES 

002  C040601M  (M  ) 

1 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0148 

DESCRIPTION: 

12TH  GRADERS 

IN 

EXTRACURR 

ACTS 

IN  SCIENCE? 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040602 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040602Y  (01  1 

1 0 

YES 

002  C040602M  (M  1 

1 1 

MISSING 

CONDITIONING  VARIABLE' ID: 

SCHL0149 

DESCRIPTION: 

12TH  GRADERS 

IN 

EXTRACURR 

ACTS 

IN  ENG/LANG  ARTS? 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040603 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C040603Y  (01 

) 0 

YES 

002  C040603M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 

SCHL0150 

2 

1 


2 

1 


2 

1 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

12TH  GRADERS  IN  EXTRACURR  ACTS  IN  ARTS? 
N12 

NAEP  ID; 

TYPE  OF  CONTRAST; 

C040604  TOTAL  NUMBER  OF  SPECIFIED  ciONTRASTS : 2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  C040604Y  (01 

002  C040604M  (M 

) 0 YES 

) 1 MISSING 

conditioning  variable  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0151 

12TH  GRADERS  IN  EXTRACURR  ACTS  IN  NONE  OF  ABOVE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040605  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS : 1 

001  C040605Y  (01 

002  C040605M  (M 

) 0 YES 

) 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0152 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  MATH? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST; 

C040701  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C040701Y  (01  ] 

002  C040701M  (M  ] 

1 0 yes 

1 1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0153 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  SCIENCE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040702  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

■ CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C040702Y  (01  ) 

002  C040702M  (M  ) 

' 0 yes 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0154 

12TH  GRADERS  IN  SUMMER  PORGRAMS  IN  ENG/LANG  ARTS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040703  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS : 1 

001  C040703Y  (01  ) 

002  C040703M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

SCHL0155 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  ARTS? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040704  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C040704Y  (01  ) 

002  C040704M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

SCHL0156 

12TH  GRADERS  IN  SUMMER  PROGRAMS  IN  NONE  OF  ABOVE? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

CO 4 0705  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  2 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  C040705Y  (01  ) 

002  C040705M  (M  ) 

0 YES 

1 MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 

SCHL0157 

# STUDS  ENROLLED  IN  AP  SCIENCE  COURSES? 
N12 

NAEP  ID: 

TYPE  OF  CONTRAST: 

C040801  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  8 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS:  7 

001  C040801A  (01  ) 

002  C040801B  (02  ) 

003  C040801C  (03  ) 

004  C040801D  (04  ) 

005  C040801E  (05  ) 

006  C040801F  (06  ) 

007  C040801G  (07  ) 

008  C040801M  (M  ) 

0000000  NONE 

1000000  1-10 

0100000  11-25 

0010000  26-50 

0001000  51-75 

0000100  76-99 

0000010  100  OR  MORE 

0000001  missing 

conditioning  variable  ID: 
DESCRIPTION: 

SCHL0158 

# STUDS  ENROLLED  IN  AP  CALC:ULUS  COURSES? 

0^*8 

ERIC 

oco 

Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


grades /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST: 

C040802 

CLASS 

001  C040802A 

(01  ) 

1 0000000 

002  C040802B 

(02  ) 

1 1000000 

003  C040802C 

(03  ) 

1 0100000 

004  C040802D 

(04  ) 

1 0010000 

005  C040802E 

(05  ) 

1 0001000 

006  C040802F 

(06  1 

1 0000100 

007  C040802G 

(07  ] 

1 0000010 

008  C040802M 

(M  ] 

1 0000001 

CONDITIONING 

VARIABLE  ID: 

SCHL0159 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

1-10 

11-25 

26-50 

51-75 

76-99 

100  OR  MORE 
MISSING 


DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


# STUDS  ENROLLED  IN  AP  COMP  SCI  COURSES? 
N12 


C040803 


TYPE  OF  CONTRAST: 

CLASS 

001  C040803A  (01 

) 

0000000 

002  C040803B  (02 

) 

1000000 

003  C040803C  (03 

) 

0100000 

004  C040803D  (04 

) 

0010000 

005  C040803E  (05 

) 

0001000 

006  C040803F  (06 

) 

0000100 

007  C040803G  (07 

) 

0000010 

008  C040803M  (M 

) 

0000001 

CONDITIONING  VARIABLE  ID: 

SCHL0160 

DESCRIPTION: 

# STUDS  El 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040804 

TYPE  OF  CONTRAST: 

CLASS 

001  C040804A  (01 

) 

0000000 

002  C040804B  (02 

) 

1000000 

003  C040804C  (03 

) 

0100000 

004  C040804D  (04 

) 

0010000 

005  C040804E  (05 

) 

0001000 

006  C040804F  (06 

) 

0000100 

007  C040804G  (07 

) 

0000010 

008  C040804M  (M 

) 

0000001 

CONDITIONING  VARIABLE 

ID: 

SCHL0161 

DESCRIPTION: 

ANY  12TH  ' 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040901 

TYPE  OF  CONTRAST: 

CLASS 

001  C040901Y  (01 

) 

1 0 

002  C040901M  (M 

• ) 

1 

CONDITIONING  VARIABLE 

ID: 

SCHL0162 

DESCRIPTION: 

ANY  12TH 

GRADES /ASSESSMENTS ; 

N12 

CONDITIONING  VAR  LABEL 

NAEP  ID: 

C040902 

TYPE  OF  CONTRAST: 

CLASS 

001  C040902Y  (01 

1 

1 0 

002  C040902M  (M 

\ 1 

CONDITIONING  VARIABLE 

ID: 

SCHL0163 

DESCRIPTION: 

ANY  12TH 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040903 

TYPE  OF  CONTRAST: 

CLASS 

001  C040903Y  (01 

) 0 

002  C040903M  (M 

) 1 

CONDITIONING  VARIABLE 

ID: 

SCHL0164 

DESCRIPTION: 

ANY  12TH 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

C040904 

TYPE  OF  CONTRAST: 

CLASS 

001  C040904Y  (01 

) 0 

002  C040904M  (M 

) 1 . 

CONDITIONING  VARIABLE 

ID: 

SCHL0165 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

NONE 

1-10 

11-25 

26-50 

51-75 

76-99 

100  OR  MORE 
MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

NONE 

1-10 

11-25 

26-50 

51-75 

76-99 

100  OR  MORE 
MISSING 


TOTAL' NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 

YES 

MISSING 


8 

7 


8 

7 


8 

7 


2 

1 


2 

1 


2 

1 


2 

1 


BEST  COPY  AVAjUBLE 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 


ANY  12TH  GRDS  TAKING  COLLEGE  COURSES  IN  NONE  ABOVE 
N12 

C040905  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

CLASS  NUMBER  OF  INDEPENDENT  CONTRASTS: 


001  C040905Y  (01 

) 0 

YES 

002  C040905M  (M 

) 1 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

SCHL0166 

WHAT  % 12TH  GRDS  HELD  BACK 

AND 

REPEAT  12TH  GRADE? 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

N12 

C041001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C041001A  (01 

) 00000 

0% 

002  C041001B  (02 

) 10000 

1-2% 

003  C041001C  (03 

) 01000 

3-5% 

004  C041001D  (04 

) 00100 

6-10% 

005  C041001E  (05  ; 

1 00010 

MORE  THAN  10% 

006  C041001M  (M  ; 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

SCHL0167 

LAST  YR  WHAT  % OF  12TH  GRDS  GRADUATED? 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

C041101 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C041101A  (01  ] 

1 00000 

99-100% 

002  C041101B  (02  ) 

1 10000 

95-98% 

003  C041101C  (03  ] 

1 01000 

90-94% 

004  C041101D  (04  ] 

1 00100 

75-89% 

005  C041101E  (05  ) 

1 00010 

LESS  THAN  75% 

006  C041101M  (M  ) 

1 00001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

SCHL0168 

WHAT  % OF  GRADUATING  CLASS 

NOW 

IN  2-YR  COLLEGE? 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

N12 

C036001 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036001A  (01  ) 

000000 

0-10% 

002  C036001B  (02  ) 

100000 

11-25% 

003  C036001C  (03  ) 

010000 

26-50% 

004  C036001D  (04  ) 

001000 

51-75% 

005  C036001E  (05  ) 

000100 

76-90% 

006  C036001F  (06  ) 

000010 

91-100% 

007  C036001M  (M  ) 

000001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

SCHL0169 

WHAT  % OF  GRADUATING  CLASS 

NOW 

IN  4-YR  COLLEGE? 

GRADES /ASSESSMENTS : 

N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

C036002 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036002A  (01  ) 

000000 

0-10% 

002  C036002B  (02  ) 

100000 

11-25% 

003  C036002C  (03  ) 

010000 

26-50% 

004  C036002D  (04  ) 

001000 

51-75% 

005  C036002E  (05  ) 

000100 

76-90% 

006  C036002F  (06  ) 

000010 

91-100% 

007  C036002M  (M  ) 

000001 

MISSING 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

SCHL0170 

WHAT  % OF  GRADUATING  CLASS 

NOW 

IN  VO-TEC  SCHOOL? 

GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 

N12 

C036003 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 

TYPE  OF  CONTRAST: 

CLASS 

NUMBER  OF  INDEPENDENT  CONTRASTS: 

001  C036003A  (01  ) 

000000 

0-10% 

002  C036003B  (02  ) 

100000 

11-25% 

003  C036003C  (03  ) 

010000 

26-50% 

004  C036003D  (04  ) 

001000 

51-75% 

005  C036003E  (05  ) 

000100 

76-90% 

006  C036003F  (06  ) 

000010 

91-100% 

007  C036003M  (M  ) 

000001 

MISSING 

2 

1 


6 

5 


6 

5 


7 

6 


7 

6 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


SCHL0171 

WHAT  % OF  GRAD  CLASS  NOW  IN  EMPLOYER  TRAINING? 

N12 

C036004  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


7 


O'~o 

ERIC 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST: 


001  C03 

002  COS 

003  COS 

004  COS 

005  COS 

006  COS 

007  COS 


6004A  (01 
6004B  (02 
6004C  (03 
6004D  (04 
6004E  (05 
6004F  (06 
6004M  (M 


CLASS 

) 000000 
) 100000 
) 010000 
) 001000 
) 000100 
) 000010 
) 000001 


NUMBER  OF  INDEPENDENT  CONTRASTS: 


0-10% 

11-25% 

26-50% 

51-75% 

76-90% 

91-100% 

MISSING 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 
NAEP  ID: 


SCHL0172 

% OF  GRADUATING  CLASS  NOW  IN  MILITARY  SERVICE? 
N12 


C036005 


TYPE  OF  CONTRAST: 

CLASS 

001  C036005A  (01  ) 

000000 

002  C036005B  (02  ) 

100000 

003  C036005C  (03  ) 

010000 

004  C036005D  (04  ) 

001000 

005  C036005E  (05  ) 

000100 

006  C036005F  (06  ) 

000010 

007  C036005M  (M  ) 

000001 

CONDITIONING  VARIABLE  ID: 

SCHL0173 

DESCRIPTION: 

PERCENT  OF  STUDENTS 

WHO 

GRADES /ASSESSMENTS: 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS,  SOS, 

N12 

NAEP  ID: 

NTLUNSC 

TYPE  OF  CONTRAST: 

CLASS 

001  NATLUNCM  (M  ) 

0 

002  NATLUNCH  (@  ) 

1 

CONDITIONING  VARIABLE  ID: 

SCHL0174 

DESCRIPTION: 

PERCENT  OF  STUDENTS 

WHO 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS,  SOS, 

N12 

NAEP  ID: 

NTLUNSC 

TYPE  OF  CONTRAST: 

LINEAR 

001  NATLUNCL  (0-100, M=0  ) 

0.0  + 1.0*X 

CONDITIONING  VARIABLE  ID: 

SCHL0175 

DESCRIPTION: 

PERCENT  OF  STUDENTS 

WHO 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS,  SOS, 

N12 

NAEP  ID: 

REMRDSC 

TYPE  OF  CONTRAST: 

CLASS 

001  REMREADM  (M  ) 

0 

002  REM  READ  (@  ) 

1 

CONDITIONING  VARIABLE  ID: 

SCHL0176 

DESCRIPTION: 

PERCENT  OF  STUDENTS 

WHO 

GRADES/ASSESSMENTS:  ■ 

N04,  S04,  NOS,  SOS, 

N12 

CONDITIONING  VAR  LABEL: 

NAEP  ID: 

REMRDSC 

TYPE  OF  CONTRAST: 

LINEAR 

001  REMREADL  (0-100, M=0  1 

1 0.0  + 1.0 *X 

CONDITIONING  VARIABLE  ID: 

SCHL0177 

DESCRIPTION: 

PERCENT  OF  STUDENTS 

WHO 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS,  SOS, 

N12 

NAEP  ID: 

REMMHSC 

TYPE  OF  CONTRAST: 

CLASS 

001  REMMATHM  (M 

) 0 

002  REM  MATH  (@ 

) 1 

CONDITIONING  VARIABLE  ID: 

SCHL017S 

DESCRIPTION: 

PERCENT  OF  STUDENTS.  WHO 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04,  NOS,  SOS, 

N12 

NAEP  ID: 

REMMHSC 

TYPE  OF  CONTRAST: 

LINEAR 

001  REMMATHL  (0-100, M=0 

) 0.0  + 1.0*X 

CONDITIONING  VARIABLE  ID: 

SCHL0179 

DESCRIPTION: 

PERCENT  OF  STUDENTS  IN 

GRADES /ASSESSMENTS : 
CONDITIONING  VAR  LABEL: 

N04,  S04 

NAEP  ID: 

NTLUNGR 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

0-10% 

11-25% 

26-50% 

51-75% 

76-90% 

91-100% 

MISSING 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


MISSING 

PERCENT 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MISSING 

PERCENT 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 


LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS: 

MISSING 

PERCENT 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 


ERIC 


672 
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Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


TYPE  OF  CONTRAST; 


CLASS 


NUMBER  OF  INDEPENDENT  CONTRASTS;  1 


001  NATLUN4M  (M  } Q 

002  NATLUN4  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0180 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N04,  S04 

NTLUNGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  1 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 . 


001  NATLUN4L  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0181 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N04,  S04 


REMRDGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


2 

1 


001  REMREA4M  (M  ) 0 

002  REMREAD4  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID: 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0182 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N04,  S04 

REMRDGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  REMREA4L  ( 0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0183 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N04,  S04 


REMMHGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  REMMAT4M  (M  ) Q 

002  REMMATH4  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID: 

TYPE  OF  CONTRAST; 


SCHL0184 

PERCENT  OF  STUDENTS  IN  4TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N04,  S04 

TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


REMMHGR 

LINEAR 


001  REMMAT4L  (0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


1 

1 


2 

1 


1 

1 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0185 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 


NTLUNGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

NUMBER  OF  INDEPENDENT  CONTRASTS;  1 


001  NATLUN8M  (M  } Q 

002  NATLUN8  (0  ) 1 


MISSING 

PERCENT 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0186 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 


NTLUNGR 

LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS;  1 

NUMBER  OF  INDEPENDENT  CONTRASTS;  1 


001  NATLUN8L  ( 0-100, M=0  ) 0.0  + 1.0*X 


LINEAR 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION: 
GRADES/ASSESSMENTS : 
CONDITIONING  VAR  LABEL; 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0187 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N08,  S08 


REMRDGR 

CLASS 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS; 
NUMBER  OF  INDEPENDENT  CONTRASTS; 


001  REMREA8M  (M  ) 0 

002  REMREAD8  (@  ) l 


MISSING 

PERCENT 


2 

1 


CONDITIONING  VARIABLE  ID; 
DESCRIPTION; 

GRADES /ASSESSMENTS ; 
CONDITIONING  VAR  LABEL: 
NAEP  ID; 

TYPE  OF  CONTRAST; 


SCHL0188 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N08,  S08 


REMRDGR 

LINEAR 


TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS: 
NUMBER  OF  INDEPENDENT  CONTRASTS : 


1 

1 


Table  C-5  (continued) 

Conditioning  Variables  Specific  to  the  1996  Science  Assessment 


001  REMREA8L  ( 0-100, M=0  ) 

0.0  + 1.0*X  LINEAR 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0189 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N08,  S08 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

REMMHGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  REMMAT8M  (M  ] 

002  REMMATH8  (0  ] 

1 0 MISSING 

1 1 PERCENT 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0190 

PERCENT  OF  STUDENTS  IN  8TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N08,  S08 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

REMMHGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  REMMAT8L  ( 0-100, M=0 

) 0.0  + 1.0*X  LINEAR 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 

SCHL0191 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

NTLUNGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  NATLU12M  (M 

002  NATLUN12  (0 

) 0 MISSING 

) 1 PERCENT 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES/ ASSESSMENTS : 

SCHL0192 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  THE  NATIONAL  SCHOOL  LUNCH  PROGRAM 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

NTLUNGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  NATLU12L  (0-100, M=0 

) 0.0  + 1.0*X  LINEAR 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0193 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

REMRDGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  number  of  INDEPENDENT  CONTRASTS:  1 

001  REMRD12M  (M 

002  REMRD12  (0 

) 0 MISSING 

) 1 PERCENT 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 
GRADES/ASSESSMENTS : 

SCHL0194 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  READING 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

REMRDGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  REMRD12L  (0-100, M=0 

) 0.0  + 1.0 *X  LINEAR 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0195 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

REMMHGR  TOTAL  NUMBER  OF  SPECIFIED  CONTRASTS:  2 

class  NUMBER  OF  INDEPENDENT  CONTRASTS:  1 

001  REMMH12M  (M 

002  REMMH12  (0 

) 0 MISSING 

) 1 PERCENT 

CONDITIONING  VARIABLE  ID: 
DESCRIPTION: 

GRADES /ASSESSMENTS : 

SCHL0196 

PERCENT  OF  STUDENTS  IN  12TH  WHO  PARTICIPATED  IN  REMEDIAL  MATH 
N12 

CONDITIONING  VAR  LABEL: 
NAEP  ID: 

TYPE  OF  CONTRAST: 

r£MmHgR  total  NUMBER  OF  SPECIFIED  CONTRASTS:  1 

LINEAR  NUMBER  OF  INDEPENDENT  CONTRASTS;  1 

001  REMMH12L  (0-100, M=0 

) 0.0  + 1.0*X  LINEAR 

674 


623 


Table  C-6 

Conditioning  Variables  for  the  Long-Term  Trend  Reading  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable  Coding 

Contrast 

Coding 

Overall 

All 

— 

1 

Gender 

All 

DSEX 

Male 

0 

Female 

1 

Region 

All 

REGION 

Northeast 

000 

Southeast 

100 

Central 

010 

West 

001 

Parental  Education 

All 

PARED 

Less  than  high  school 

0000 

High  school  graduate 

1000 

Post-high  school 

0100 

College  graduate 

0010 

Missing  and  I don’t  know 

0001 

Items  in  the  Home 

All 

B000901 

None  of  the  six  items 

00 

B000902 

One  of  the  six  items 

10 

B000903 

Two  of  the  six  items 

20 

B000904 

Three  of  the  six  items 

30 

B000905 

Four  of  the  six  items 

40 

B000906 

Five  of  the  six  items 

50 

Six  of  the  six  items 

60 

Missing 

01 

Television 

All 

BOO 1801 

None 

00 

Watching 

One  hour  or  less 

10 

Two  hours 

20 

Three  hours 

30 

Four  hours 

40 

Five  hours 

50 

Six  or  more  hours 

60 

Missing 

01 

Homework 

All 

B001701 

Don’t  have  any 

00 

Don’t  do  any. 

00 

Less  than  1 hour 

10 

1-2  hours 

20 

More  than  2 hours 

30 

Missing 

01 

Language  Spoken 

All 

B000401 

English 

00  - 

at  Home 

Spanish 

10 

Other 

10 

Missing 

01 

ERIC 


24 


675 


Table  C-6  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Reading  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable  Coding 

Contrast 

Coding 

Pages  Read 

All 

BOOllOl 

More  than  20 

10 

16-20 

10 

11-15 

10 

6-10 

10 

5 or  fewer 

00 

Missing 

01 

Percent  in  School 

All 

C032001 

None 

00000000 

Lunch  Program 

1-5% 

10000000 

6-10% 

01000000 

11-25% 

00100000 

26-50% 

00010000 

51-75% 

00001000 

76-90% 

00000100 

over  90% 

00000010 

Missing 

00000001 

Percent  White 

All 

PCTWHTQ 

0-49% 

100 

50-79% 

010 

80-100% 

001 

Missing 

000 

Derived 

All 

DRACE 

White 

000 

Race/Ethnicity 

Black 

100 

Hispanic 

010 

Asian  American 

001 

American  Indian 

000 

Unclassified 

000 

Missing 

000 

Age  by  Grade 

All 

MODGRAG 

< age,  = grade 

0000 

= age,  < grade 

1000 

= age,  = grade 

0100 

= age,  > grade 

0010 

>■  age,  = grade 

0001 

School  Type 

All 

SCHTYPE 

Public 

1 

• 

Private,  Catholic,  Bureau  of 

0 

Indian  Affairs,  Department  of 

Defense 

Description  of 

All 

DOC 

Big  City 

0000 

Community 

Fringe 

1000 

Medium  City 

0100 

Small  Place 

0010 

Missing 

0001 

ERIC 


676 


625 


Table  C-6  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Reading  Assessment  in  1996 


Conditioning 

Age 

Variable 

Contrast 

Variable 

Classes 

Name(s) 

Variable  Coding 

Coding 

Type  of  Location 

All 

TOL8 

Big  City 

00000000 

Medium  City 

10000000 

Fringe  of  Big  City 

01000000 

Fringe  of  Medium  City 

00100000 

Large  Town 

00010000 

Small  Place 

00001000 

Rural  - MSA 

00000100 

Rural  - Non  MSA 

00000010 

Missing 

00000001 

Courses  Taken 

9,  13 

BOOlOOl 

None 

00 

B001002 

1 

10 

B001003 

2 

20 

B001004 

3 

30 

B001005 

4 

40 

B001006 

5 

50 

B001007 

6 

60 

7 

70 

Missing 

01 

O 626 

ERIC 


67 


Table  C-7 

Conditioning  Variables  for  the  Long-Term  Trend  Mathematics  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable  Coding 

Contrast 

Coding 

Overall 

All 

— 

1 

Gender 

All 

GENDER 

Male 

0 

Female 

1 

Observed 

All 

ETHNIC 

White 

000 

Race/Ethnicity 

Black 

100 

Hispanic 

010 

Asian  American 

001 

American  Indian 

000 

Other 

000 

Missing 

000 

Size  and  Type  of 

All 

STOC 

1,  4-7  all  except  2 and  3 

01 

Community 

2 Low  Metro 

00 

3 High  Metro 

10 

Region 

All 

REGION 

Northeast 

000 

Southeast 

100 

Central 

010 

West 

001 

Parents’ 

All 

PARED 

Less  than  high  school 

0000 

Education 

High  school  graduate 

1000 

Post-high  school 

0100 

College  graduate 

0010 

Missing  and  I Don't  Know 

0001 

Modal  Grade 

All 

MODGRD 

< modal  grade 

10 

= modal  grade,  missing 

00 

> modal  grade 

01 

Items  in  the  Home 

All 

HOMEEN2 

0 to  2 items 

00 

(of  newspaper,  > 

3 items 

10 

25  books. 

4 items 

01 

encyclopedia. 

magazines) 

673 


627 


Table  C-7  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Mathematics  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable  Coding 

Contrast 

Coding 

Observed 

All 

RACEx 

White,  male 

— Q ^ 

000 

Race/Ethnicity  by 

GENDER 

Black,  male 

000 

Gender  (“White” 

Hispanic,  male 

000 

includes 

Asian  American,  male 

000 

American  Indian 

White,  female 

000 

and  Other) 

Black,  female 

100 

Hispanic,  female 

010 

Asian  American,  female 

001 

Observed 

9 

RACEx 

White,  < HS 

0000  0000  0000 

Race/Ethnicity  by 

PARED 

White,  HS  graduate 

0000  0000  0000 

Parents’ 

White,  post-HS 

0000  0000  0000 

Education 

White,  college  grad. 

0000  0000  0000 

(“White”  includes 

White,  missing 

0000  0000  0000 

American  Indian 

Black,  < HS 

0000  0000  0000 

and 

Black,  HS  grad  & post-HS 

1000  0000  0000 

Other) — coded 

Black,  college  grad. 

0010  0000  0000 

differently  for 

Black,  missing 

0001  0000  0000 

each  age  class 

Hispanic,  < HS 

0000  0000  0000 

Hispanic,  HS  grad  & post-HS 

0000  1000  0000 

Hispanic,  coll.  grad. 

0000  0010  0000 

Hispanic,  missing 

0000  0001  0000 

Asian  Amer.,  < HS 

0000  0000  0000 

Asian  Amer.,  HS  grad  & post-HS 

0000  0000  1000 

Asian  Amer.,  coll.  grad. 

0000  0000  0010 

Asian  Amer.,  missing 

0000  0000  0001 

O ^28 


G7;) 


Table  C-7  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Mathematics  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable  Coding 

Contrast 

Coding 

Observed 

13 

RACEx 

White,  < HS 

0000  0000  0000 

Race/Ethnicity  by 

PARED 

White,  HS  graduate 

0000  0000  0000 

Parents’ 

White,  post-HS 

0000  0000  0000 

Education 

White,  college  grad. 

0000  0000  0000 

(“White”  includes 

White,  missing 

0000  0000  0000 

American  Indian 

Black,  < HS 

0000  0000  0000 

and 

Black,  HS  graduate 

1000  0000  0000 

Other) — coded 

Black,  post-HS 

0100  0000  0000 

differently  for 

Black,  college  grad. 

0010  0000  0000 

each  age  class 

Black,  missing 

0001  0000  0000 

Hispanic,  < HS 

0000  0000  0000 

Hispanic,  HS  grad. 

0000  1000  0000 

Hispanic,  post-HS 

0000  0100  0000 

Hispanic,  coll.  grad. 

0000  0010  0000 

Hispanic,  missing 

0000  0001  0000 

Asian  Amer.,  < HS 

0000  0000  0000 

Asian  Amer.,  HS  grad. 

0000  0000  1000 

Asian  Amer.,  post-HS 

0000  0000  0100 

Asian  Amer.,  coll.  grad. 

0000  0000  0010 

Asian  Amer.,  missing 

0000  0000  0001 

Observed 

17 

RACE  X 

White,  < HS 

0000  0000  0000 

Race/Ethnicity  by 

PARED 

White,  HS  graduate 

0000  0000  0000 

Parents’ 

White,  post-HS 

0000  0000  0000 

Education 

White,  college  grad. 

0000  0000  0000 

(“White”  includes 

White,  missing 

0000  0000  0000 

American  Indian 

Black,  < HS 

0000  0000  0000 

and 

Black,  HS  graduate 

1000  0000  0000 

Other) — coded 

Black,  post-HS 

0100  0000  0000 

differently  for 

Black,  college  grad. 

0010  0000  0000 

each  age  class 

Black,  missing 

0001  0000  0000 

Hispanic,  < HS 

0000  0000  0000 

Hispanic,  HS  grad. 

0000  1000  0000 

Hispanic,  post-HS 

0000  0100  0000 

Hispanic,  coll.  grad. 

0000  0010  0000 

Hispanic,  missing 

0000  0001  0000 

Asian  Amer.,  < HS 

0000  0000  0000 

Asian  Amer.,  HS  grad. 

00000000  1000 

Asian  Amer.,  post-HS,  coll.  grad.  0000  0000  0100 

Asian  Amer.,  missing 

0000  0000  0001 

Language  in  the 

All 

LANGHOM 

Never 

00 

Home 

Sometimes 

10 

Always 

01 

Observed  Race  by 

All 

RACEx 

White,  often 

00  00  00 

629 


Table  C-7  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Mathematics  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable  Coding 

Contrast 

Coding 

Language  at 

LANGHOM 

White,  sometimes 

00  00  00 

Home 

White,  never 

00  00  00 

Black,  often  and  sometimes 

10  00  00 

Black,  often 

10  00  00 

Black,  sometimes 

01  00  00 

Black,  never 

00  00  00 

Hispanic,  often  and  sometimes 

00  10  00 

Hispanic,  often 

00  10  00 

Hispanic,  sometimes 

00  01  00 

Hispanic,  never 

00  00  00 

Asian  American,  often  and  sometimes 

00  00  10 

Asian  American,  often 

00  00  10 

Asian  American,  sometimes 

00  00  01 

Asian  American,  never 

00  00  00 

Derived 

All 

DRACE 

White 

000 

Race/Ethnicity 

Black 

100 

Hispanic 

010 

Asian  American 

001 

Other 

000 

Missing 

000 

Homework 

13,  17 

HW 

None  assigned 

100 

Didn’t  do 

010 

1/2  hour  or  less 

012 

1 hour 

013 

2 hours 

014 

More  than  2 hours 

000 

Missing 

000 

Highest  Level  of 

17 

NMATH 

Pre-Algebra 

10000 

Mathematics 

Algebra 

01000 

Class 

Geometry 

00100 

Algebra  2 

00010 

Calculus 

00001 

Something  else 

00000 

High  School 

17 

HS_PGM 

General 

00 

Program 

College  Preparatory 

10 

V ocational/Technical 

01 

Missing 

00 

ERIC 


681 


Table  C-7  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Mathematics  Assessment  in  1996 


Conditioning 


Age 


Variable 


Contrast 


VariaDie 

Type  of  Location  All  TOL8 

Big  City 

00000000 

J L 

Medium  City 

10000000 

Fringe  of  Big  City 

01000000 

Fringe  of  Medium  City 

00100000 

Large  Town 

00010000 

Small  Place 

00001000 

Rural  - MSA 

00000100 

Rural  - non  MSA 

00000010 

. Missing 

00000001 

682 


631 


Table  C-8 

Conditioning  Variables  for  the  Long-Term  Trend  Science  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable 

Coding 

Contrast 

Coding 

Overall 

All 

— 

1 

Gender 

All 

DSEX 

Male 

0 

Female 

1 

Observed  Race 

All 

RACE 

White 

000 

Black 

100 

Hispanic 

010 

Asian  American 

001 

American  Indian,  Pacific  Islander 

000 

Other,  blank,  missing 

000 

Size  and  Type 

All 

STOC 

Low  Metro 

10 

of  Conimunity 

High  Metro 

y \j 

01 

(92  only) 

All  others,  missing 

00 

Type  of 

All 

TOL8 

Big  City 

00000000 

Location  (94 

Medium  City 

10000000 

and  96  only) 

Fringe  of  Big  City 

01000000 

Fringe  of  Medium  City 

00100000 

Large  Town 

00010000 

Small  Place 

00001000 

Rural  - MSA 

00000100 

Rural  - Non  MSA 

00000010 

Missing 

00000001 

Region 

All 

REGION 

Northeast 

000 

Southeast 

100 

Central 

010 

West 

001 

• 

Missing 

000 

Parents’ 

All 

PARED 

Less  than  high  school 

0000 

Education 

High  school  graduate 

1000 

Post-high  school 

0100 

College  graduate 

0010 

Missing  and  I don’t  know 

0001 

Modal  Grade 

All 

MODGRD 

< modal  grade 

10 

= modal  grade 

00 

> modal  grade 

01 

Missing 

00 

632 

er|c 


683 


Table  C-8  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Science  Assessment  in  1996 


Conditioning 
; Variable 

Age 

Classes 

Variable 

Name(s) 

Variable 

Coding 

Contrast 

Coding 

Observed  Race 

All 

RACEx 

White,  Male 

000 

by  Gender 

DSEX 

Black,  Male 

000 

Hispanic,  Male 

000 

Asian  American,  Male 

000 

White,  Female 

000 

Black,  Female 

100 

Hispanic,  Female 

010 

Asian  American,  Female 

001 

Other  combinations,  missing 

000 

Observed  Race 

All 

RACEx 

White,  < High  school 

000000000000 

by  Parents’ 

PARED 

White,  = High  school 

000000000000 

Education 

White,  > High  school 

000000000000 

White,  Graduated  college 

000000000000 

White,  missing  or  unknown 

000000000000 

Black,  < High  school 

000000000000 

Black,  = High  school 

100000000000 

Black,  > High  school 

010000000000 

Black,  Graduated  college 

001000000000 

Black,  missing  or  unknown 

000100000000 

Hispanic,  < High  school 

000000000000 

Hispanic,  = High  school 

000010000000 

Hispanic,  > High  school 

000001000000 

Hispanic,  Graduated  college 

000000100000 

Hispanic,  missing  or  unknown 

000000010000 

Asian  American,  < High  school 

000000000000 

Asian  American,  = High  school 

000000001000 

Asian  American,  > High  school 

000000000100 

Asian  American,  Graduated  college 

000000000010 

Asian  American,  missing  or 

000000000001 

unknown 

School  Type 

All 

SCHTYPE 

Public 

0 

Private,  Catholic,  BIA,  DoDEA 

1 

Missing 

0 

Items  in  the 

All 

HOMEEN2 

0-2  items 

00 

Home 

3 items 

10 

4 items 

01 

Missing 

00 

ERIC 


684 


633 


Table  C-8  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Science  Assessment  in  1996 


Conditioning 

Variable 

Age 

Classes 

Variable 

Name(s) 

Variable 

Coding 

Contrast 

Coding 

Derived  Race 

All 

DRACE 

White 

— 

000 

Black 

100 

Hispanic 

010 

Asian  American 

001 

American  Indian,  Pacific  Islander 

000 

Other,  missing 

000 

Language 

All 

LANGHO 

Never 

00 

Spoken  in  the 

M 

Sometimes 

10 

Home  (Other 

Always 

01 

than  English) 

Missing 

00 

Observed  Race 

All 

RACEx 

White,  Always 

000000 

by  Language  in 

LANGHO 

White,  Sometimes 

000000 

the  Home 

M 

White,  Never 

000000 

Black,  Always 

100000 

Black,  Sometimes 

010000 

Black,  Never 

000000 

Hispanic,  Always 

001000 

Hispanic,  Sometimes 

000100 

Hispanic,  Never 

000000 

Asian  American,  Always 

000010 

Asian  American,  Sometimes 

000001 

Asian  American,  Never 

000000 

One  or  both  missing 

000000 

Homework 

13,  17 

BOO 1701 

None  assigned 

100 

Didn’t  do 

010 

1/2  hour  or  less 

012 

One  hour 

013 

Two  hours 

014 

More  than  two  hours 

000 

Missing 

000 

Highest  Level 

17 

B005308 

General  Science 

1000 

of  Science 

B005309 

Biology 

0100 

Class 

B005310 

Chemistry 

0010 

B005311 

Physics 

0001 

Nothing,  Something  Else 

0000 

Missing 

0000 

High  School 

17 

B005001 

General 

00 

Program 

College  Preparatory 

10 

Vocational,  technical 

01 

Missing 

00 

Table  C-9 

Conditioning  Variables  for  the  Long-Term  Trend  Writing  Assessment  in  1996 


Conditioning 

Variable 

Variable 

Number  of 
Contrasts 

Contrasts 

Contrasts 

Codes 

Intercept 

1 

Overall 

1 

Gender 

DSEX 

1 

Male 

0 

Female 

1 

Race/Ethnicity 

Ethnic(84) 

2 

Black 

00 

BRACE  (88,90,92,94,96) 

Hispanic 

01 

White+others 

10 

Type  of 

STOC 

2 

STOC: 

Community/ 

(84,88,90,92) 

Low  Metro 

00 

Location 

Everyone  else 

01 

Hi  Metro 

10 

TOL8(94,96) 

7 

TOL8: 

Large  City  & miss 

0000000 

Medium  City 

1000000 

Urb.  Fring  of  LC 

0100000 

Urb.  Fring  of  MC 

0010000 

Big  Town 

0001000 

Small  Town 

0000100 

Rural-MSA 

0000010 

Rural-non-MSA 

0000001 

Region 

REGION 

3 

Northeast 

000 

Southeast 

100 

Central 

010 

West 

001 

Parent's  Education 

PARED 

4 

Less  than  H.S. 

0000 

High  School 

1000 

Greater  HS 

0100 

College 

0010 

I don’t  know 

0001 

Items  in  the  home 

B000901,  B000902, 

2 

Articles  in  home: 

B000903,  B000904 

0-3  or  missing 

00 

B000905 

4 

10 

5 

01 

Modal  age 

DAGE 

2 

Modal  Age 

00 

< Modal  age 

10 

> Modal  age 

01 

ERIC 
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Table  C-9  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Writing  Assessment  in  1996 


Conditioning 

Variable 

Variable 

Number  of 
Contrasts 

Contrasts 

Contrasts 

Codes 

Homework 

BOO 1701 

5 

None  assigned 

00000 

None  done 

10000 

< 1 hr 

01000 

1-2  hrs 

00100 

2+  hrs 

00010 

Missing  HW 

00001 

School  type 

SCHTYPE 

2 

Public  School 

0 

Non-Public 

1 

Missing 

1 

TV  watching. 

BOO 1801 

1 

None 

0 

Linear 

1 hour 

1 

2 hours 

2 

3 hours 

3 

Missing 

3 

4 hours 

4 

5 hours 

5 

6 hours 

6 

TV  watching. 

BOO 1801 

2 

None 

00 

Quadratic 

1 hour 

01 

2 hours 

04 

3 hours 

09 

Missing 

09 

4 hours 

16 

5 hours 

25 

6 hours 

36 

Mother  Work 

B000801 

1 

Mother  does  not  work  Missing 

0 

Outside  the  Home 

Mother  works 

0 

1 


Language  Minority 

B000301 

2 

Not  language  minority 

00 

(self  speaking  at 

Language  minority 

10 

home) 

Missing 

01 

Percent  in  Lunch 

C001301(84) 

7 

None  in  lunch  program 

0000000 

Program 

PCLUNCH  (88,90) 

1-5% 

1000000 

C0320101(92,94,96) 

6-10% 

0100000 

11-25% 

0010000 

26-50% 

0001000 

51-75% 

0000100 

76-90% 

0000010 

over  90% 

0000001 

Minority  School 

PCTWHT 

2 

Minority  (0-49.9%) 

00 

"6" 

ERIC 
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Table  C-9  (continued) 

Conditioning  Variables  for  the  Long-Term  Trend  Writing  Assessment  in  1996 


Conditioning 

Variable 

Variable 

Number  of 
Contrasts 

Contrasts 

Contrasts 

Codes 

PCTWHTQ 

Intergrated  (50-  79.9%) 

10 

' 

White  or  Missing  (80-100%) 

01 

Grades  in  School 

B001901 

1 Var 

Grade 

0.0-4.0 

(Missing  included  at  2.0) 

Pages  Read  for  HW 

BOOllOl 

3 

No  pages  read 

000 

11+  pages  read 

100 

6-10  pages  read 

010 

5 or  less  pages  read 

001 

Number  of  reports 

B001201-7  and  B001208 

1 

Number  of  reports,  essays  etc. 

0-7 

O 

ERIC 


683 
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Table  C-10 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  4 


Proportion  of 


Contrast 

Variance 

FEMALE 

0 . 92422 

BLACK 

0.93928 

HISPANIC 

0 . 96987 

ASIAN 

0 . 94278 

MEXICAN 

0.95600 

PUER  RIC 

0.92864 

CUBN,OTH 

0 . 96742 

HISP-? 

0 . 74847 

NON  MSA 

0.92046 

MID  CTY5 

0 . 94280 

FR/BTWN5 

0.94911 

SML  TWN5 

0.95067 

RURAL 5 

0.95097 

URBAN  FR 

0.90811 

MED  CITY 

0 . 89044 

SM  PLACE 

0.93298 

HS  GRAD 

0 . 94671 

POST  HS 

0.92986 

COL  GRAD 

0.92976 

PARED- ? 

0.94023 

S EAST 

0.89683 

CENTRAL 

0.89537 

WEST 

0.90228 

PRIVATE 

0.93637 

CATHOLIC 

0.91800 

lEP-NO 

0 . 87421 

LEP-NO 

0.79025 

CHAPl-N 

0.77000 

RED  PRIC 

0.93333 

FREE 

0.73534 

INFO  N/A 

0.85427 

SCH/REF 

0.84591 

HL-SOME 

0.85950 

HL-ALWAY 

0.85486 

HL-? 

0.80373 

B008901N 

0.95242 

B008901M 

0.70149 

TVLIN-0 

0.98291 

TV-QUAD 

0.98263 

HW-NO 

0.96512 

HW-YES 

0.96865 

HWLIN-0 

0.97548 

HWQUAD- 0 

0.96240 

HITEM=3 

0.86215 

HITEM=4 

0.82846 

MOMHOM-N 

0.81244 

MOMHOM- ? 

0.88256 

DADHOM-N 

0.86602 

DADHOM- ? 

0.92245 

MISS-2< 

0.93929 

USA  >5 

0.93754 

USA  3-5 

0.92960 

USA  <3 

0.92448 

USA-? 

0.83469 

STGRDl-2 

0.90017 

Proportion  of 

Contrast  Variance 


STGRD3> 

0.88466 

SNRM-LIN 

0.88196 

SCHCHG-1 

0.91473 

SCHCHG-2 

0.91001 

SCHCHG-3 

0.81984 

DIS0HOM2 

0.89416 

DIS0HOM3 

0.93408 

DIS0HOM4 

0.79632 

PGS>5 

0.82911 

PGS>10 

0.83882 

G/R  22 

0.91016 

G/R  23 

0.90786 

G/R  24 

0.95898 

G/T  22 

0.71798 

G/T  23 

0.73468 

G/T  24 

0.77888 

G/T  25 

0.73060 

G/P  22 . 

0.92747 

G/P  23 

0.89300 

G/P  24 

0.85988 

G/P  25 

0.84783 

G/S  22 

0.93396 

G/S  23 

0.89897 

R/T  24 

0.89235 

R/T  25 

0.90331 

R/T  31 

0.90034 

R/T  32 

0.89554 

R/T  33 

0.88687 

R/T  34 

0.88052 

R/T  35  ■ 

0.85371 

R/T  41 

0.88419 

R/T  42 

0.92435 

R/T  43 

0.93164 

R/T  44 

0.93998 

R/T  45 

0.94866 

R/P  24 

0.89509 

R/P  25 

0.87983 

R/P  31 

0.90064 

R/P  32 

0.91138 

R/P  33 

0.89862 

R/P  34 

0.90463 

R/P  35 

0.89655 

R/P  41 

0.89841 

R/P  42 

0 . 96949 

R/P  43 

0.96354 

R/P  44 

0.96174 

R/P  45 

0 . 96426 

R/S  31 

0.97978 

R/S  32 

0.96311 

R/S  33 

0.97170 

R/S  41 

0.95920 

R/S  42 

0.96764 

R/S  43 

0.96940 

T/P  25 

0.81392 

T/P  31 

0.82723 
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Contrast 

Proportion  of 
Variance 

T/P  32 

0.79005 

T/P  33 

0.79097 

T/P  34 

0.78458 

T/P  35 

0.80476 

T/P  41 

0.76880 

T/P  42 

0.74497 

T/P  43 

0.77272 

T/P  44 

0.80053 

T/P  45 

0.83715 

T/P  51 

0.83797 

T/P  52 

0.77254 

T/P  53 

0.81563 

T/P  54 

0.79556 

T/P  55 

0.79026 

T/S  32 

0.95336 

T/S  33 

0.93082 

T/S  41 

0.93927 

T/S  42 

0.93128 

T/S  43 

0.96264 

T/S  51 

0.94403 

T/S  52 

0.94797 

P/S  32 

0.96888 

P/S  33 

0.94519 

P/S  41 

0.95888 

P/S  42 

0.93031 

P/S  43 

0.95356 

P/S  51 

0.92917 

P/S  52 

0.95848' 

P/S  53 

0.93447 

SAMP  S2 

0.87473 

/R  23 

0.90811 

/R  24 

0.90508 

/R  31 

0.96289 

B009301B 

0.95836 

B009301C 

0.96313 

B009301D 

0.96067 

B009301E 

0.94948 

B009301M 

0.92458 

B009401B 

0.94274 

B009401C 

0.94860 

B009401D 

0 . 90675 

B009401M 

0.91517 

B009501B 

0.95808 

B009501C 

0.84231 

B009501M 

0.89573 

B009502B 

0.95167 

B009502C 

0.86065 

B009502M 

0.87799 

M812701B 

0 . 87446 

M812701C 

0.92214 

M812701D 

0 . 84959 

M812701M 

0.75474 

M812702B 

0.87726 

M812702C 

0.90130 

M812701D 

0.87129 

Table  C-10  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

M812701M 

M812101F 

0.77048 

M812101G 

M812703B 

0 . 94466 

M812101M 

M812703C 

0.95361 

M811401N 

M812703D 

0.93975 

M811401M 

M812703M 

0.71661 

M811101B 

M812704B 

0.93276 

M811101C 

M812704C 

0.93105 

M811101M 

M812704D 

0.92633 

M811103B 

M812704M 

0.72479 

M811103C 

M812705B 

0.92230 

M811103M 

M812705C 

0 . 92740 

M811106B 

M812705D 

0 . 90764 

M811106C 

M812705M 

0.76198 

M811106M 

M812706B 

0 . 92303 

M811109B 

M812706C 

0 . 93229 

M811109C 

M812706D 

0 . 93410 

M811109M 

M812706M 

0.72774 

M811107B 

M812707B 

0.91670 

M811107C 

M812707C 

0.93858 

M811107M 

M812707D 

0.88953 

M811105B 

M812707M 

0.89200 

M811105C 

M812708B 

0.88593 

M811105M 

M812708C 

0.89736 

M811108B 

M812708D 

0 . 87920 

M811108C 

M812708M 

0.68806 

M811108M 

M812709B 

0.91900 

M811110B 

M812709C 

0 . 92927 

M811110C 

M812709D 

0 . 90477 

M811110M 

M812709M 

0.72509 

MMOOIOIB 

M812710B 

0.92641 

MMOOIOIC 

M812710C 

0.94339 

MMOOIOID 

M812710D 

0.9.0871 

MMOOIOIM 

M812710M 

0.84702 

MM00201B 

M812711B 

0.93730 

MM00201C 

M812711C 

0.95239 

MM00201D 

M812711D 

0.92672 

MM00201M 

M812711M 

0.84435 

MM00301B 

M811201N 

0.85363 

MM00301C 

M811201M 

0.84968 

MM00301D 

M812001B 

0.91257 

MM00301M 

M812001C 

0.93958 

MM00401B 

M812001D 

0.89292 

MM00401C 

M812001M 

0.78811 

MM00401D 

M812002B 

0.93992 

MM00401M 

M812001C 

0.94042 

MM00501B 

M812001D 

0.89765 

MM00501C 

M812001M 

0.81152 

MM00501D 

M812003B 

0.93589 

MM00501M 

M812003C 

0.95728 

C030901B 

M812003D 

0.93863 

C030901C 

M812003M 

0.87599 

C030901M 

M812101B 

0.95122 

C037101B 

M812101C 

0.98021 

C037101C 

M812101D 

0.98464 

C037101N 

M812101E 

0.98150 

C037101M 

O 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.97657 

C031212B 

0 . 89289 

0.95779 

C031212M 

0 . 97017 

0.77945 

C031205B 

0.90185 

0.83560 

■ C031205C 

0 . 89277 

0.81227 

C031205M 

0 . 93941 

0.90471 

C031213M 

0 . 95475 

0.85283 

C031214B 

0 . 90767 

0.84889 

C031214C 

0 . 92306 

0.90634 

C031214D 

0.90531 

0.85451 

C031214E 

0.90683 

0.84217 

C031214M 

0.92600 

0.91234 

C031603N 

0.87846 

0.90506 

C031603M 

0 . 92434 

0.75855 

C031607N 

0.89303 

0.87823 

C031607M 

0.94545 

0.85178 

C031601N 

0.88729 

0.85089 

C031601M 

0.94662 

0.83958 

C031610N 

0.90678 

0.81346 

C031610M 

0.95321 

0.85408 

C031606N 

0.88964 

0.94083 

C031606M 

0.94862 

0.92457 

C035701N 

0.88805 

0 . 85277 

C035701M 

0.93004 

0.90576 

C035702N 

0.89130 

0.88138 

C035702M 

0.92471 

0.88722 

C035703N 

0.88402 

0.93917 

C035703M 

0.92740 

0.94704 

C037201M 

0.94474 

0.87218 

C037202M 

0.92251 

0.84889 

C037207M 

0.91288 

0.86647 

C037204M 

0.90320 

0.90934 

C037205M 

0.90800 

0.88111 

C037206M 

0.93973 

0.90505 

C037301M 

0.97484 

0.90559 

C037302M 

0.96190 

0.90476 

C037303M 

0.96240 

0.91399 

C037304M 

0.89266 

0.88918 

C037305M 

0.92732 

0.85485 

C037401M 

0.89938 

0.90776 

C037402M 

0.89873 

0.84424 

C037403M 

0.88274 

0.92763 

C037404M 

0.88499 

0.94108 

C03.7405M 

0.88798 

0.94031 

C037406M 

0.87276 

0.90467 

C037501M 

0.87432 

0.88779 

C037502M 

0.88889 

0.89769 

C037503M 

0.88022 

0.91306 

C037504M 

0.89411 

0.80670 

C037505M 

0.91016 

0.87546 

C037601M 

0.91900 

0.89822 

C037602M' 

0.90388 

0.97693 

C037603M  . 

0.93754 

0.89290 

C037604M 

0.89697 

0.90595 

C037605M 

0 . 92835 

0.94737 

C036601B 

0.90340 

0.93370 

C036601C 

0.94582 
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Table  C-10  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

C036601D 

0.96106 

C032411C 

0.92916 

T056001C 

0 . 85497 

C036601M 

0.93323 

C032411D 

0.95921 

T056001D 

0 . 88643 

C032207B 

0.90906 

C032412C 

0.90880 

T056001E 

0.89026 

C032207N 

0.88328 

C032413B 

0.87343 

T056001F 

0.90057 

C032207M 

0 . 92796 

C032413C 

0.89923 

T056001M 

0.93437 

C032209B 

0.88240 

C032414B 

0.91543 

T040301B 

0.90023 

C032209N 

0.91232 

C032414C 

0.95362 

T040301C 

0.91623 

C032209M 

0 . 97717 

C032414D 

0.93445 

T040301D 

0.92629 

C032210B 

0.89290 

C032415B 

0.91543 

T040301E 

0.93485 

C032210N 

0.89959 

C032415C 

0.88831 

T040301M 

0.97112 

C032210M 

0.94318 

C032502B 

0.90525 

T056101B 

0.91949 

C032211B 

0.89806 

C032502C 

0.91341 

T056101C 

0.92407 

C032211N 

0 . 87633 

C032502D 

0.91998 

T056101D 

0.93806 

C032211M 

0.97839 

C032503B 

0.89030 

T056101E 

0 . 94465 

C037701B 

0 . 88757 

C032503C 

0.90449 

T056101M 

0 . 97687 

C037701C 

0 . 89656 

C032505B 

0.89514 

T056102B 

0.88880 

C037701D 

0 . 90758 

C032505C 

0.91082 

T056102C 

0.89683 

C037701M 

0.98981 

C032505D 

0.90382 

T056102D 

0.91719 

C037702B 

0.90859 

C032506B 

0.88183 

T056102E 

0.92978 

C037702C 

0.93760 

C032506C 

0.89775 

T056102M 

0.90207 

C037702D 

0 . 94255 

C032506D 

0.94373 

T056201B 

0 . 87996 

C037703B 

0 . 91776 

C033601B 

0.88746 

T056201C 

0 . 87424 

C037703C 

0.92911 

C033601C 

0.91444 

T056201D 

0.90152 

C037703D 

0.95420 

C033601M 

0.96982 

T056201E 

0.89091 

C037704B 

0.89812 

C036501B 

0.88549 

T056201F 

0 . 86307 

C037704C 

0 . 88562 

C036501C 

0.88945 

T056201M 

0.89431 

C037704D 

0 . 87868 

C037801B 

0.91251 

T040501N 

0 . 86646 

C037704M 

0 . 94449 

C037801C 

0.91425 

T040501C 

0..  87847 

C037705B 

0.90509 

C037801D 

0.91956 

T040501M 

0.88168 

C037705C 

0.89905 

C037801E 

0.89582 

T040506N 

0.91930 

C037705D 

0.89193 

C037801F 

0.88578 

T040506C 

0.93518 

C037705M 

0.98275 

C037801G 

0.89624 

T040506M 

0.92416 

C032402B 

0.91690 

C037801H 

0.87801 

T040504N 

0 . 92875 

C032402C 

0.93603 

C037801M 

0.93805  . 

T040504C 

0 .91604 

C032402D 

0.93000 

C037901B 

0.90039 

T040504M 

0 . 93770 

C032402M 

0.95263 

C037901C 

0.88338 

T040507N 

0.92414 

C032401B 

0.92205 

C037901E 

0.89721 

T040507C 

0.93028 

C032401C 

0.94109 

C038001B 

0.88267 

T040507M 

0.93216 

C032401D 

0.94183 

C038001C 

0.89724 

T040508N 

0.92610 

C032401M 

0.96538 

C038001D 

0.90367 

T040508C 

0.92511 

C032404B 

0.91540 

C038001F 

0.91884 

T040508M 

0.95049 

C032404C 

0.94872 

C038001G 

0.89475 

T040505N 

0.87425 

C032404D 

0.93668 

C038301N 

0.87255 

T040505C 

0.85036 

C032404M 

0.97495 

C038801N 

0.90074 

T040505M 

0.88310 

C032406B 

0.89929 

C034101M 

0.90931 

T056301B 

0.86863 

C032406C 

0.89908 

C034102M 

0.86390 

T056301C 

0.94542 

C032407B 

0.93072 

C034103M 

0.90034 

T056301D 

0.94757 

C032407C 

0.90227 

C034104M 

0.91039 

T056301E 

0.87991 

C032408B 

0.92336 

C034105M 

0.88246 

T056301F 

0.89665 

C032408C 

0.93971 

C034106M 

0.87927 

T056301G 

0.92785 

C032408D 

0.94243 

C034107M 

0.89607 

T040701M 

0.85850 

C032409B 

0.90573 

C034108M 

0.88519 

T040706M 

0.87270 

C032409C 

0.94692 

C034109M 

0.90284 

T040707M 

0.88064 

C032409D 

0.93129 

T055901B 

0 . 83449 

T040703M 

0.85761 

C032410B 

0.90989 

T055901M 

0.95592 

T040704M 

0.89422 

C032410C 

0.91409 

T056001B 

0.85919 

T040710M 

0.86605 

Er|c  6'9i 


Table  C-10  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

T040711M 

0.88088 

T058201E 

T040712M 

0.88793 

T058201M 

T040713M 

0.89131 

T056701M 

T040708M 

0.86839 

T056702M 

T040709M 

0.88842 

T056703M 

T040705M 

0.86609 

T056704M 

T040801M 

0 . 86782 

T056705M 

T040807M 

0 . 88407 

T056706M 

T040808M 

0.87263 

T056707M 

T040803M 

0 . 85837 

T056708M 

T040804M 

0.87242 

T056709M 

T040814M 

0.86338 

T056710M 

T040815M 

0.89963 

T056711M 

T040816M 

0.90897 

T056712M 

T040817M 

0.90343 

T056713M 

T040809M 

0.85225 

T041201B 

T040810M 

0.88623 

T041201C 

T040811M 

0.86569 

T041201D 

T040812M 

0.86161 

T041201M 

T040813M 

0.87130 

T041302N 

T040805M  . 

0.87488 

T041302M 

T040806M 

0.87139 

T041303N 

T056401M 

0.87217 

T041303M 

T056402M 

0.88232 

T056801B 

T056403M 

0.86561 

T056801C 

T056404M 

0.86030 

T056801D 

T056405M 

0.85499 

T056801E 

T056413M 

0.84907 

T056801F 

T056414M 

0.87428 

T056801M 

T056415M 

0.88588 

T056901M 

T056416M 

0.88361 

T0569A1M 

T056406M 

0.84066 

T0569B1M 

T056407M 

0.87904 

T0569C1M 

T056408M 

0.86924 

T056902M 

T056409M 

0.85604 

T0569A2M 

T056410M 

0.87048 

T0569B2M 

T056411M 

0.88352 

T0569C2M 

T056412M 

0.88974 

T056903M 

T056501B 

0.91310 

T0569A3M 

T056501C 

0.91126 

T0569B3M 

T056501D 

0.89844 

T0569C3M 

T056501E 

0.87955 

T056904M 

T056501M 

0.94460 

T0569A4M 

T058101B 

0.87513 

T0569B4M 

T058101C 

0.88453 

T0569C4M 

T058101D 

0.88429 

T056905M 

T058101E 

0.85574 

T0569A5M 

T058101M 

0.94762 

T0569B5M 

T056601B 

0.88151 

T0569C5M 

T056601C 

0.87588 

T056906M 

T056601D 

0 . 89212 

T0569A6M 

T056601E 

0.87477 

T0569B6M 

T056601M 

0 . 95900 

T0569C6M 

T058201B 

0 . 88402 

T056907M 

T058201C 

0.87801 

T0569A7M 

T058201D 

0 . 86875 

T0569B7M 

BEST  COPY  AVAILABLE 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.85271 

T0569C7M 

0.91690 

0.93699 

T056908M^ 

0.88151 

0.84442 

T0569A8M 

0.87210 

0.86785 

T0569B8M 

0.87636 

0.84733 

T0569C8M 

0.90877 

0.84667 

T057001N 

0.86053 

0.86584 

T057001M 

0.90995 

0.88206 

T057002N 

0.86184 

0.84208 

T057002M 

0.93314 

0.86398 

T057003N 

0.84736 

0.86342 

T057003M 

0-.93251 

0.85226 

T057004N 

0.84355 

0.84552 

T057004M 

0.92756 

0.86144 

T057005N 

0.83569 

0.85136 

T057005M 

0.93750 

0.91372 

T057006N 

0.85345 

0.91852 

T057006M 

0.92040 

0.87742 

T057007N 

0.86267 

0.97165 

T057007M 

0.91579 

0.92005 

T057101B 

0.88236 

0.90450 

T057101C 

0.92509 

0.91591 

T057101D 

0.93133 

0.89679 

T057101M 

0.92361 

0.88290 

T057201M 

0.87624 

0.91737 

T057211M 

0.85779 

0.93092 

T057221M 

0.87096 

0.90584 

T057231M 

0.87670 

0.91298 

T057241M 

0.91-534 

0.92515 

T057301B 

0.87060 

0.87090 

T057301C 

0.87623 

0.88752 

T057301M 

0 . 95025 

0.84911 

T057302B 

0.87815 

0.87488 

T057302M 

0.95258 

0.90372 

T057303B 

0 . 87830 

0.90806 

T057303C 

0.90424 

0.86996 

T057303M 

0.95719 

0.88831 

T057304B 

0 . 84929 

0.88404 

T057304C 

0.86139 

0.90967 

T057304M 

0 . 92002 

0.90105 

T057401B 

0.90239 

0.87855 

T057401C 

0 . 91147 

0.88116 

T057401D 

0.91456 

0.89535 

T057401M 

0 . 94497 

0.88101 

T057402B 

0 . 87718 

0.89052 

T057402C 

0 . 86257 

0.88390 

T057402D 

0 . 86972 

0.89367 

T057402M 

0 . 87157 

0.86742 

T057403B 

0 . 89626 

0.88765 

T057403C 

0.90086 

0.91177 

T057403D 

0 . 87718 

0.89529 

T057403M 

0.95064 

0.86272 

T057404B 

0 . 89499 

0.91056 

T057404C 

0.90763 

0.90241 

T057404D 

0.89079 

0.89949 

T057404M 

0 . 96581 

0.88021 

T057405B 

0.90667 

641 
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Contrast 

T057405C 

T057405D 

T057405M 

T057501B 

T057501C 

T057501D 

T057501E 

T057501F 

T057501M 

T057601B 

T057601C 

T057601D 

T057601E 

T057601M 

T044002N 

T044002M 

T057701B 

T057701C 

T057701D 

T057701M 

T057801B 

T057801C 

T057801D 

T057801E 

T057801M 

T044201N 

T044201M 

T044301B 

T044301C 

T044301M 

T057901B 

T057901C 

T057901D 

T057901M 

T044401B 

T044401C 

T044401D 

T044401E 

T044401F 

T044401M 

T044501B 

T044501C 

T044501D 

T044501M 

T044502B 

T044502C 

T044502D 

T044502M 

T044512B 

T044512C 

T044512D 

T044513B 

T044513C 

T044513D 

T044513M 

T044514B 


Table  C-10  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  4 


Proportion  of 
Variance 

0.88626 

0.89650 

0.96396 

0.90647 

0.90728 

0.88542 

0.90756 

0.88592 

0.88922 

0.90068 

0.87531 

0.87089 

0.88152 

0.88177 

0.96057 

0.98118 

0.88013 

0.93030 

0.90493 

0.93232 

0.89626 

0.90313 

0 . 88796 

0.88363 

0.92501 

0.84915 

0.95404 

0.92323 

0.92658 

0.94428 

0.89811 

0.92830 

0.92205 

0.95806 

0.93553 

0.93908 

0.85349 

0 . 84742 

0.86032 

0.94802 

0 . 86534 

0 . 87699 

0.88415 

0.98882 

0.88635 

0.88693 

0 . 87010 

0 . 98776 

0 . 89116 

0.90369 

0.86574 

0.90891 

0.91592 

0.87999 

0.98443 

0.88894 


Contrast 


T044514C 

T044514D 

T044514M 

T044505B 

T044505C 

T044505D 

T044505M 

T044515B 

T044515C 

T044515D 

T044515M 

T044507B 

T044507C 

T044507D 

T044507M 

T044516B 

T044516C 

T044516D 

T044516M 

T044508B 

T044508C 

T044508D 

T044508M 

T044509B 

T044509C 

T044509D 

T044509M 

T044510B 

T044510C 

T044510D 

T044510M 

T044506B 

T044506C 

T044506D 

T044506M 

T058001B 

T058001C 

T058001D 

T058001M 

T058002B 

T058002C 

T058002D 

T058002M 

T058003B 

T058003C 

T058003D 

T058003M 

T058004B 

T058004C 

T058004D 

T058004M 

T058005B 

T058005C 

T058005D 

T058005M 

T058006B 


Proportion  of 
Variance 

0.93670 

0.91131 

0.94879 

0.90978 

0.93496 

0.90303 

0.96128 

0.94914 

0.95660 

0.87350 

0.97925 

0.90786 

0.90997 

0.90189 

0.94931 

0.86404 

0.87541 

0.86403 

0.97424 

0 . 87910 

0.94909 

0.94682 

0.97046 

0.88251 

0 . 87048 

0 . 87755 

0.98858 

0.88589 

0.89079 

0.87292 

0 . 97816 

0 . 91074 

0.91055 

0.89806 

0 . 98601 

0 . 87914 

0.94285 

0.86668 

0.98340 

0.86930 

0.90760 

0.88591 

0.98362 

0.89322 

0.90914 

0.87573 

0.96909 

0.91433 

0 . 91976 

0.89295 

0.94642 

0.91308 

0.92420 

0.90207 

0.96135 

0.87168 


Contrast 


T058006C 

T058007B 

T058007C 

T058008B 

T058008C 

T058008D 

T058009B 

T058009C 

T058009D 

T058009M 

T045401N 

T045401M 

T044801N 

T044801M 

T045001N 

T045001M 

T044901N 

T044901M 

T045304B 

T045304C 

T045304M 

T045305B 

T045305C 

T045305M 

T045302B 

T045302C 

T045302D 

T045303B 

T045303C 

T045303D 

T045303M 

T044000B 

T044000C 

T044000D 

T044000E 

T044000M 

NATLUNCH 

NATLUNCL 

REM  READ 

REMREADL 

REM  MATH 

REMMATHL 

NATLUN4 

NATLUN4L 

REMREAD4 

REMREA4L 

REMMATH4 

REMMAT4L 


Proportion  of 
Variance 

0.87947 
0.84998 
0.89035 
0.86926 
0.87852 
0.88313 
0 . 87471 
0.88656 
0.86106 
0.97804 
0.84916 
0.97164 
0.85058 
0.94684 
0.85883 
0.97507 
0.87216 
0.95233 
0.89609 
0.90295 
0.99250 
0.90586 
0.91145 
0.98072 
0.90421 
0.90917 
0.88258 
0.89142 
0.87448 
0.87792 
0.98069 
0.88559 
0.90168 
0.88654 
0.86796 
0.90723 
0.88867 
0.92919 
0.93790 
0.93711 
0.94580 
0.93332 
0.91489 
0.92700 
0.93241 
0 . 91492 
0.93880 
0.89187 


Table  C-11 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  8 


Proportion  of 


Contrast 

Variance 

Contrast 

FEMALE 

0.94701 

SCHCHG-2 

BLACK 

0.94977 

SCHCHG-3 

HISPANIC 

0.97892 

DIS@HOM2 

ASIAN 

0.94296 

DlS@HOM3 

MEXICAN 

0.95099 

DIS@HOM4 

PUER  RIC 

0.93906 

PGS>5 

CUBN.OTH 

0.95269 

PGS>10 

HISP-? 

0.93864 

8TH  GRD 

NON  MSA 

0.94366 

PREALG 

MID  CTY5 

0.95565 

ALGEBRA 

FR/BTWN5 

0.96058 

INT/SEQ 

SML  TWN5 

0.96035 

APPLIED 

RURAL5 

0.96816 

OTHER 

URBAN  FR 

0.92480 

MISSING 

MED  CITY 

0.89703 

G/R  22 

SM  PLACE 

0.94156 

G/R  23 

HS  GRAD 

0.96304 

G/R  24 

POST  HS 

0.95738 

G/T  22 

COL  GRAD 

0.95319 

G/T  23 

PARED- ? 

0.96129 

G/T  24 

S EAST 

0.92396 

G/T  25 

CENTRAL 

0.91036 

G/P  22 

WEST 

0.91239 

G/P  23 

PRIVATE 

0.93569 

G/P  24 

CATHOLIC 

0.93388 

G/P  25 

lEP-NO 

0.79397 

G/S  22 

LEP-NO 

0.77834 

G/S  23 

CHAPI-N 

0.78653 

R/T  24 

RED  PRIC 

0.89841 

R/T  25 

FREE 

0.75139 

R/T  31 

INFO  N/A 

0.86504 

R/T  32 

SCH/REF 

0.90436 

R/T  33 

HL-SOME 

0.84902 

R/T  34 

HL-ALWAY 

0 . 81695 

R/T  35 

HL-? 

0 . 83754 

R/T  41 

B00890IN 

0.94253 

R/T  42 

B00890IM 

0.90926 

R/T  43 

TVLIN-0 

0.98142 

R/T  44 

TV- QUAD 

0 . 97999 

R/T  45 

HW-NO 

0.96362 

R/P  24 

HW-YES 

0.96561 

R/P  25 

HWLIN-0 

0.93806 

R/P  31 

HWQUAD-0 

0 . 91809 

R/P  32 

HITEM=3 

0.89183 

R/P  33 

HITEM=4 

0.86408 

R/P  34 

MOMHOM-N 

0.84023 

R/P  35 

MOMHOM-? 

0.87898 

R/P  41 

DADHOM-N 

0 . 84817 

R/P  42 

DADHOM-? 

0.88099 

R/P  43 

MISS-2< 

0.91536 

R/P  44 

USA  >5 

0.91828 

R/P  45 

USA  3-5 

0.92784 

R/S  31 

USA  <3 

0.87446 

R/S  32 

USA-? 

0.79211 

R/S  33 

SNRM-LIN 

0.89715 

R/S  41 

SCHCHG-1 

0.92894 

R/S  42 

O 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.92875 

R/S  43 

0.95780 

0.86997 

,T/P  25 

0.80094 

0.89291 

T/P  31 

0.80134 

0.91848 

T/P  32 

0.81521 

0.82593 

T/P  33 

0.84046 

0 . 81303 

T/P  34 

0.78677 

0 . 81267 

T/P  35 

0.79144 

0 . 94151 

T/P  41 

0.78907 

0.93329 

T/P  42 

0 . 84007 

0 . 92511 

T/P  43 

0.78337 

0.93387 

T/P  44 

0.76401 

0.89124 

T/P  45 

0.81980 

0.93801 

T/P  51 

0.78691 

0.93013 

T/P  52 

0.79266 

0 . 90755 

T/P  53 

0 . 80917 

0.91042 

T/P  54 

0.82290 

0 . 95593 

T/P  55 

0 . 84808 

0.72733 

T/S  32 

0 . 93164 

0.72892 

T/S  33 

0.92003 

0.74182 

T/S  41 

0.95479 

0.74657 

T/S  42 

0 . 92872 

0.88531 

T/S  43 

0 . 95063 

0.86071 

T/S  52 

0 . 95197 

0.79421 

P/S  32 

0 . 95164 

0.81323 

P/S  33 

0.93344 

0.93783 

P/S  41 

0.94493 

0.91741 

P/S  42 

0.92261 

0.87374 

P/S  43 

0.93863 

0.88210 

P/S  51 

0.92295 

0.89950 

P/S  52 

0.95461 

0.92058 

P/S  53 

0.93576 

0.87576 

G/  22 

0.93592 

0.87096 

G/  23 

0 . 95116 

0.87505 

G/  24 

0.93179 

0.90554 

G/  25 

0.86323 

0.92224 

G/  26 

0 . 85715 

0.91507 

G/  27 

0 . 91950 

0.91470 

G/  28 

0 . 84763 

0.95898 

R/  24 

0.92793 

0.88729 

R/  2 5 

0.91838 

0.89254 

R/  2 6 

0 . 91243 

0.88749 

R/  27 

0.89314 

0.86224 

R/  28 

0.89928 

0.89687 

R/  31 

0 . 87915 

0.90150 

R/  32 

0.86721 

0.89567 

R/  33 

0.92894 

0.87712 

R/  34 

0.92274 

0.95636 

R/  35 

0.92208 

0.95964 

R/  36 

0.90532 

0.94623 

R/  37 

0.90052 

0.92191 

R/  38 

0.88622 

0.96606 

R/  41 

0.87288 

0.96618 

R/  42 

0.95356 

0.96676 

0.94469 

0.97132 

643 


694 


43 

44 

45 

46 

47 

48 

25 

26 

27 

28 

31 

32 

33 

34 

35 

36 

37 

38 

41 

42 

43 

44 

45 

46 

47 

48 

51 

52 

53 

54 

55 

56 

57 

58 

25 

26 

27 

28 

31 

32 

33 

34 

35 

36 

37 

38 

41 

42 

43 

44 

45 

46 

47 

48 

51 

52 


Table  C-11  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  8 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

0.95586 

T/  53 

0.83497 

M812709B 

0.94588 

T/  54 

0.84404 

M812709C 

0.89045 

T/  55 

0.87899 

M812709D 

0.87926 

T/  57 

0.81430 

M812709M 

0.92260 

S/  23 

0.95571 

M812710B 

0.93993 

S/  24 

0.94874 

M812710C 

0 . 91725 

S/  25 

0.93655 

M812710D 

0.89966 

S/  26 

0.87380 

M812710M 

0.88288 

S/  27 

0.88821 

M812711B 

0.81238 

S/  28 

0.94125 

M812711C 

0.82703 

S/  31 

0.88571 

M812711D 

0.78422 

S/  32 

0.94201 

M812711M 

0.81740 

S/  33 

0.93769 

M811201N 

0.88873 

S/  34 

0.92429 

M811201M 

0.86981 

S/  37 

0.96185 

M812001B 

0.92637 

SAMP  S2 

0.87564 

M812001C 

0.82100 

/R  23 

0.90500 

M812001D 

0.84099 

/R  24 

0.89817 

M812001M 

0.77060 

/R  31 

0.95481 

M812002B 

0.81732 

B009301B 

0.94197 

M812001C 

0.87637 

B009301C 

0.95406 

M812001D 

0.85636 

B009301D 

0.95267 

M812001M 

0.83856 

B009301E 

0.93658 

M812003B 

0.79113 

B009301M 

0.82239 

M812003C 

0.87264 

B009401B 

0.85769 

M812003D 

0.76559 

B009401C 

0.89311 

M812003M 

0.80307 

B009401D 

0.87783 

M812101B 

0.87510 

B009401M 

0.83927 

M812101C 

0.83157 

M812701B 

0.90965 

M812101D 

0.89636 

M812701C 

0.93011 

M812101E 

0.80892 

M812701D 

0.84236 

M812101F 

0.80434 

M812701M 

0.82648 

M812101G 

0.82456 

M812702B 

0.89768 

M812101M 

0.81977 

M812702C 

0.91660 

MMOOIOIB 

0.86093 

M812701D 

0.90267 

MMOOIOIC 

0.84930 

M812701M 

0.82565 

MMOOIOID 

0.82121 

M812703B 

0.92509 

MMOOIOIM 

0.87708 

M812703C 

0.93064 

MM00201B 

0.85837 

M812703D 

0.92087 

MM00201C 

0.80234 

M812703M 

0.69206 

MM00201D 

0.80283 

M812705B 

0.94087 

MM00201M 

0.85945 

M812705C 

0.94574 

MM00301B 

0.81043 

M812705D 

0.92095 

MM00301C 

0.82767 

M812705M 

0.78616 

MM00301D 

0.86522 

M812706B 

0.94530 

MM00301M 

0.87206 

M812706C 

0.95627 

MM00401B 

0.77217 

M812706D 

0.90033 

MM00401C 

0.82045 

M812706M 

0.74296 

MM00401D 

0.87237 

M812707B 

0.90126 

MM00401M 

0.84620 

M812707C 

0.91977 

MM00501B 

0.86059 

M812707D 

0.86700 

MM00501C 

0.95946 

M812707M 

0.88850 

MM00501D 

0.88866 

M812708B 

0.87445 

MM00501M 

0.81806 

M812708C 

0.89525 

B009701B 

0.85272 

M812708D 

0.83380 

B009701C 

0.88552 

M812708M 

0.74980 

B009701D 

O 


Proportion  of 
Variance 

0.88785 
0.90083 
0.86335 
0.74053 
0.93236 
0.93526 
0.91073 
0.79454 
0.86971 
0.89593 
0.84064 
0.85688 
0.86266 
0.80875 
0.84867 
0.89470 
0.82194 
0.81513 
0.89876 
0.90771 
0.86499 
0.91680 
. 0.87811' 
0.92138 
0.89682 
0 . 87967 
0.88739 
0.95249 
0.98305 
0.96073 
0.94860 
0.94817 
0.76705 
0.90473 
0.90502 
0 . 92171 
0 . 84443 
0.95161 
0 . 95178 
0 . 94556 
0.88655 
0.92159 
0.92104 
0.92272 
0 . 83735 
0.90564 
0.90656 
0.90258 
0 . 85979 
0.91546 
0.91780 
0.91964 
0.86315 
0.90685 
0.89817 
0.93126 


Table  C-11  (continued)  , 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  8 


Contrast 

Proportion  of 
Variance 

B009701E 

0.92208 

B009701F 

0.95144 

B009701M 

0.84595 

GRAD  HS 

0.94110 

ED  > HS 

0.96475 

GRAD  CLG 

0.96336 

GRAD  SCH 

0.94283 

MISSING 

0.77302 

B009601B 

0.87176 

B009601C 

0.84997 

B009601D 

0.84804 

B009601M 

0.88366 

B009601B 

0.94583 

B009601C 

0.92200 

B009601D 

0.85322 

B009601M 

0.86797 

M812712B 

0.94893 

M812712C 

0.95875 

M812712D 

0.95137 

M812712M 

0.72729 

M812713B 

0.95233 

M812713C 

0.97003 

M812713D 

0.96448 

M812713M 

0.77091 

M812201N 

0.89140 

M812201M 

0.76400 

M812301N 

0.81021 

M812301M 

0.83552 

M812401N 

0.82627 

M812401M 

0.84167 

BASIC 

0.93725 

APPLIED 

0.96260 

PREALG 

0.92809 

ALG  1 

0.95522 

GEOMETRY 

0.78999 

INT/SEQ 

0.81907 

OTHER 

0.88223 

IDK 

0.95595 

MISSING 

0.80969 

M812701B 

0.91250 

M812701C 

0.92949 

M812701D 

0.95011 

M812701E 

0.85028 

M812701M 

0.85388 

M810701B 

0.88582 

M810701C 

0.93173 

M810701D 

0.94071 

M810701E 

0 . 81516 

M810701M 

0.81252 

M810707B 

0.87195 

M810707C 

0.90383 

M810707D 

0.92895 

M810707E 

0.80457 

M810707M 

0.76770 

M810709B 

0.96430 

M810709C 

0.97372 

O 


Contrast 

Proportion  of 
Variance 

M810709D 

0.97616 

M810709E 

0.97474 

M810709M 

0.87758 

M810708B 

0.95046 

M810708C 

0 . 94811 

M810708D 

0.95470 

M810708E 

0.95704 

M810708M 

0.84154 

M810710B 

0.90331 

M810710C 

0.90910 

M810710D 

0.95931 

M810710E 

0.91239 

M810710M 

0.87203 

M810705B 

0.86691 

M810705C 

0.89354 

M810705D 

0.91930 

M810705E 

0.90340 

M810705M 

0.92837 

M810706B 

0.94797 

M810706C 

0.94923 

M810706D 

0.96217 

M810706E 

0.92694 

M810706M 

0.92244 

M810711B 

0.88507 

M810711C 

0.89834 

M810711D 

0.91593 

M810711E 

0.90353 

M810711M 

0.90112 

C031603N 

0.91594 

C031603M 

0.96515 

C031607N 

0.92005 

C031607M 

0.96658 

C031610N 

0.90820 

C031610M 

0.94926 

C031606N 

0.89310 

C031606M 

0.95348 

C035701N 

0.90366 

C035701M 

0.94218 

C035702N 

0.88445 

C035702M 

0.97011 

C035703N 

0.88549 

C035703M 

0.92685 

C037201M 

0.94973 

C037202M 

0.93131 

C037204M 

0.90177 

C037205M 

0.8925.6 

C037206M 

0.92693 

C037301M 

0.97405 

C037302M 

0.96113 

C037304M 

0.90304 

C037305M 

0.93896 

C036601B 

0.89148 

C036601C 

0.91413 

C036601D 

0.94539 

C036601M 

0.97685 

C032207B 

0.91227 

Contrast 

Proportion  of 
Variance 

. C032207N 

0.93248 

C032207M 

0.93447 

C032209B 

0.90029 

C032209N 

0.90276 

C032209M 

0.98585 

C032210B 

0.92826 

C032210N 

0.92751 

C032210M 

0.95062 

C032211B 

0.90726 

C032211N 

0.88456 

C032211M 

0.98648 

C037701B 

0.90614 

C037701C 

0.91168 

C037701D 

0.90296 

C037701M 

0.98076 

C037702B 

0.92327 

C037702C 

0.93673 

C037702D 

0.93390 

C037702M 

0.97849 

C037703B 

0.91672 

C037703C 

0.93131 

C037703D 

0.93545 

C037703M 

0.99105 

C037704B 

0.88942 

C037704C 

0 . 89140 

C037704D 

0.90698 

C037704M 

0.98275 

C037705B 

0.90535 

C037705C 

0.90174 

C037705D 

0.89648 

C037705M 

0.98646 

C032402B 

0.93467 

C032402C 

0.94533 

C032402D 

0.93473 

C032401B 

0.92963 

C032401C 

0.94813 

C032401D 

0.93644 

C032404B 

0.91664 

C032404C 

0.95186 

C032404D 

0.93068 

C032406B 

0.89857 

C032406C 

0.94637 

C032406D 

0.94673 

C032406M 

0.97844 

C032407B 

0.88564 

C032407C 

0.90939 

C032408B 

0.91070 

C032408C 

0.94679 

C032408D 

0.94800 

C032409B 

0.92746 

C032409C 

0.94160 

C032409D 

0.92181 

C032410B 

0.92980 

C032410C 

0.94784 

C032410D 

0.95246 

C032411B 

0.93020 

69G 


645 


Contrast 


C032411C 

C032412B 

C032412C 

C032413B 

C032413C 

C032413D 

C032414B 

C032414C 

C032414D 

C032415B 

C032415C 

C032502B 

C032502C 

C032503B 

C032503C 

C032503M 

C032505B 

C032505C 

C032506B 

C032506C 

C032506D 

C033601B 

C033601C 

C033601D 

C036501B 

C036501C 

C037801B 

C037801C 

C037801D 

C037801E 

C037801F 

C037801G 

C037801H 

C037801M 

C038001B 

C038001C 

C038001D 

C038001E 

C038001G 

C038301N 

C038301M 

C038801N 

C038801M 

C034101M 

C034102M 

C034103M 

C034104M 

C034105M 

C034106M 

C034107M 

C034108M 

C034109M 

T055901B 

T055901M 

T056001B 

T056001C 


Table  C-11  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  8 


Proportion  of 
Variance 


0 . 91582 
0.91253 
0.93321 
0.91076 
0.94270 
0.94985 
0.94363 
0.95104 
0.88788 
0 . 92749 
0.89179 
0.90780 
0.91345 
0 . 92169 
0.90117 
0 . 98531 
0.91602 
0 . 90715 
0 . 90674 
0 . 90228 
0 . 90674 
0 . 92819 
0 . 92044 
0.90542 
0 . 91363 
0 . 90872 
0 . 91523 
0 . 92636 
0 . 89795 
0 . 89586 
0 . 91907 
0 . 89618 
0.93549 
0 . 97520 
0.90960 
0.90497 
0 . 88774 
0.89327 
0.93207 
0.91318 
0.96584 
0 . 90484 
0.94554 
0 . 91960 
0 . 85874 
0.88720 
0 . 90990 
0 . 91509 
0 . 88247 
0 . 90550 
0 . 89866 
0.89096 
0.85163 
0 . 98717 
0.83897 
0.85430 


Proportion  of 

Contrast  Variance 


T056001D 

0 . 87634 

T056001E 

0.88429 

T056001F 

0.85685 

T056001M 

0.98103 

T040301B 

0 . 89927 

T040301C 

0.91648 

T040301D 

0 . 93642 

T040301E 

0.94637 

T040301M 

0 . 99299 

T056201B 

0 . 90194 

T056201C 

0.88094 

T056201D 

0.88681 

T056201E 

0.88529 

T056201F 

0.88692 

T056201M 

0.95836 

T040501N 

0 . 89508 

T040501C 

0 . 88914 

T040501M 

0 . 88564 

T040506N 

0.92133 

T040506C 

0 . 87177 

T040506M 

0.92065 

T040504N 

0 . 87686 

T040504C 

0.87640 

T040504M 

0.89901 

T040505N 

0 . 89973 

T040505C 

0 . 87897 

T040505M 

0 . 90676 

T056301B 

0.88169 

T056301C 

0 . 94810 

T056301D 

0.93610 

T056301E 

0 . 88520 

T056301F 

0 . 89411 

T056301G 

0 . 88805 

T056301M 

0 . 97870 

T040701M 

0.87808 

T040706M 

0 . 88877 

T040707M 

0.86265 

T040703M 

0.87527 

T040704M 

0 . 86729 

T040708M 

0.86015 

T040709M 

0 . 86686 

T040705M 

0.88113 

T040801M 

0 . 87021 

T040807M 

0 . 87467 

T040808M 

0 . 87728 

T040803M 

0.87427 

T040804M 

0.87398 

T040809M 

0.86626 

T040810M 

0.86959 

T040811M 

0.86418 

T040812M 

0.86015 

T040813M 

0.86959 

T040805M 

. 0.87448 

T040806M 

0.89637 

T056401M 

0.88013 

T056402M 

0.87195 

Contrast 

Proportion  of 
Variance 

T056403M 

0 . 86155 

T056404M 

0.89005 

T056405M 

0 . 87917 

T056406M 

0 . 86674 

T056407M 

0 . 89077 

T056408M 

0.86793 

T056409M 

0.86406 

T056410M 

0.87493 

T056411M 

0.89169 

T056412M 

0.88958 

T056501B 

0.89340 

T056501C 

0.91637 

T056501D 

0.90411 

T056501E 

0.89452 

T056601B 

0.86582 

T056601C 

0 . 85079 

T056601D 

0.87115 

T056601E 

0.87349 

T056701M 

0.86788 

T056702M 

0 . 87040 

T056703M 

0.86560 

T056704M 

0.85028 

T056705M 

0.86112 

T056706M 

0.85983 

T056707M 

0.84627 

T056708M 

0.87261 

T056709M 

0.85872 

T056710M 

0.86888 

T056711M 

0.86264 

T056712M 

0.85995 

T056713M 

0.86100 

T041201B 

0.88619 

T041201C 

0.88360 

T041201D 

0.95982 

T056801B 

0.88338 

T056801C 

0.89619 

T056801D 

0.92371 

T056801E 

0.93047 

T056801F 

0.91406 

T056801M 

0 . 97424 

T056901M 

0.90315 

T0569A1M 

0.88271 

T0569B1M 

0.87323 

T0569C1M 

0.89853 

T056902M 

0 . 90492 

T0569A2M 

0.91483 

T0569B2M 

0.89312 

T0569C2M 

0.88269 

T056903M 

0 . 90217 

T0569A3M 

0 . 89857 

T0569B3M 

0.89419 

T0569C3M 

0 . 89079 

T056904M 

0 . 89713 

T0569A4M 

0 . 88016 

T0569B4M 

0 . 89325 

T0569C4M 

0 . 87946 

Table  C-11  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  8 


Contrast 

Proportion  of 
Variance 

Contrast 

T056905M 

0.90363 

T057402D 

T0569A5M 

0.88528 

T057402M 

T0569B5M 

0.89336 

T057403B 

T0569C5M 

0 . 87814 

T057403C 

T056906M 

0.91481 

T057403D 

T0569A6M 

0.88477 

T057403M 

T0569B6M 

0.86503 

T057404B 

T0569C6M 

0.89837 

T057404C 

T056907M 

0.92663 

T057404D 

T0569A7M 

0.86584 

T057404M 

T0569B7M 

0.86403 

T057405B 

T0569C7M 

0.89780 

T057405C 

T056908M 

0.91516 

T057405D 

T0569A8M 

0.89206 

T057501B 

T0569B8M 

0.88691 

T057501C 

T0569C8M 

0.89879 

T057501D 

T057001N 

0.87328 

T057501E 

T057001M 

0.95849 

T057501F 

T057002N 

0 . 87625 

T057501M 

T057002M 

0.98859 

T057601B 

T057003N 

0.86007 

T057601C 

T057003M 

0.97643 

T057601D 

T057004N 

0.88587 

T057601E 

T057004M 

0.973i3 

T057601M 

T057005N 

0.86140 

T044002N 

T057005M 

0.95776 

T044002M 

T057006N 

0.87462 

T057701B 

T057006M 

0.94233 

T057701C 

T057007N 

0.88237 

T057701D 

T057007M 

0.95541 

T057701M 

T057101B 

0.88244 

T057801B 

T057101C 

0.91693 

T057801C 

T057101D 

0.88574 

T057801D 

T057101M 

0.94623 

T057801E 

T057201M 

0.88932 

T057801M 

T057211M 

0.84955 

T044201N 

T057221M 

0.88253 

T044201M 

T057231M 

0.85572 

T044301B 

T057241M 

0.91787 

T044301C 

T057301B 

0.86461 

T044301M 

T057301C 

0.88359 

T057901B 

T057301M 

0.98678 

T057901C 

T057302B 

0.87224 

T057901D 

T057302C 

0.86300 

T057901M 

T057303B 

0.86019 

T044401B 

T057303C 

0.86201 

T044401C 

T057303M 

0.99279 

T044401D 

T057304B 

0.85741 

T044401E 

T057304C 

0.86730 

T044401F 

T057304M 

0.98213 

T044401M 

T057401B 

0.90660 

T044501B 

T057401C 

0.90346 

T044501C 

T057401D 

0.90436 

T044501D 

T057401M 

0.97563 

T044501M 

T057402B 

0.83963 

T044502B 

T057402C 

0.85985 

T044502C 

O 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.87643 

T044502D 

0.86751 

0.91986 

T044502M 

0.98905 

0.89899 

T044512B 

0.87988 

0.90091 

T044512C 

, 0.90848 

0.90774 

T044512D 

0.88274 

0.97727 

T044512M 

0.99337 

0.90022 

T044513B 

0.89689 

0.91687 

T044513C 

0.91988 

0.88966 

T044513D 

0.89134 

0.98207 

T044513M 

0.97943 

0.88958 

T044514B 

0.86173 

0.89392 

T044514C 

0.92439 

0.90055 

T044514D 

0.93344 

0.87004 

T044514M 

0.98310 

0.87899 

T044505B 

0.86382 

0.87228 

T044505C 

0.88513 

0.89439 

T044505D 

0.88931 

0.86958 

T044505M 

0.98431 

0 . 89914 

T044515B 

0.93339 

0 . 87712 

T044515C 

0.93540 

0.87547 

T044515D 

0.88233 

0.87891 

T044515M 

0.97711 

0.89050 

T044507B 

0.90372 

0.89058 

T044507C 

0.91545 

0.94282 

T044507D 

0.89482 

0.98489 

T044507M 

0.98104 

0.80117 

T044516B 

0.88722 

0.94548 

T044516C 

0.88880 

0.83031 

T044516D 

0.86928 

0.91628 

T044516M 

0.97995 

0.88089 

T044508B 

0.84325 

0.91023 

T044508C 

0.92874 

0.86027 

T044508D 

0.94464 

0 . 85474 

T044509B 

0.87212 

0.93020 

T044509C 

0.87988 

0.83582 

T044509D 

0.85133 

0.96610 

T044509M 

0.99242 

0 . 89014 

T044510B 

0.88301 

0 . 89524 

T044510C 

0.90142 

0.99158 

T044510D 

0.88009 

0.90917 

T044506B 

0.89947 

0.92372 

T044506C 

0.90503 

0.92095 

T044506D 

0.93084 

0.98451 

T058001B 

0.83872 

0.89582 

T058001C 

0.83924 

0.93311 

T058001D 

0.90580 

0.89211 

T058001M 

0.98657 

0.88988 

T058002B 

0.88574 

0.88699 

T058002C 

0.87935 

0.98353 

T058002D 

0.84904 

0.83880 

T058002M 

0.97487 

0.87046 

T058003B 

0.87332 

0.85452 

T058003C 

0.88309 

0.98985 

T058003D 

0.85618 

0.88893 

T058003M 

0.98302 

0.89621 

T058004B 

0.88710 

647 

698 


Table  C-11  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  8 


Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

T058004C 

0.90237 

C034403N 

0.92593 

NATLUN8 

T058004D 

0.86892 

C034403M 

0.96844 

NATLUN8L 

T058004M 

0.98773 

C034401N 

0.90517 

REMREAD8 

T058005B 

0.84985 

C034401M 

0.97825 

REMREA8L 

T058005C 

0.84628 

C034406N 

0.90995 

REMMATH8 

T058005D 

0.86918 

C034406M 

0.95625 

REMMAT8L 

T058005M 

0.99108 

C034510B 

0.90143 

T058006B 

0.85669 

C034510C 

0.91646 

T058006C 

0.85238 

C034510D 

0.90488 

T058006D 

0.86313 

C034510E 

0.91600 

T058006M 

0.99161 

C034510M 

0.93872 

T058007B 

0.87286 

C034511B 

0.92490 

T058007C 

0.87980 

C034512B 

0.93015 

T058007D 

0.91941 

C034513B 

0.93065 

T058007M 

0.98868 

C034514B 

0.90435 

T058008B 

0.85248 

C034514C 

0.90408 

T058008C 

0.83660 

C034514D 

0.90448 

T058008D 

0.84409 

C034514E 

0.87394 

T058008M 

0.99088 

C034514M 

0.94466 

T058009B 

0.86469 

C031611N 

0.91608 

T058009C 

0.88677 

C031611M 

0.95481 

T058009D 

0.88230 

C034601N 

0.90328 

T058009M 

0.98684 

C034601M 

0.94237 

T045401N 

0.86512 

C037203M 

0.93229 

T045401M 

0.97901 

C037306M 

0.95528 

T044801N 

0.88113 

C039401M 

0.91790 

T044801M 

0.97237 

C039402M 

0.91270 

T045001N 

0.85439 

C039403M 

0.89748 

T045001M 

0.98589 

C039404M 

0.89834 

T044901N 

0.85653 

C039405M 

0.90496 

T044901M 

0.94842 

C039406M 

0.89760 

T045304B 

0.90464 

C039501M 

0.89080 

T045304C 

0.93677 

C039502M 

0.89988 

T045304D 

0.97602 

C039503M 

0.91084 

T045304M 

0.98756 

C039504M 

0.90820 

T045305B 

0.89497 

C039505M 

0.90996 

T045305C 

0.94552 

C039601M 

0.95724 

T045305M 

0.97247 

C039602M 

0.92269 

T045302B 

0.87919 

C039603M 

0.95032 

T045302C 

0.89584 

C039604M 

0.91266 

T045302D 

0.90152 

C039605M 

0.94171 

T045302M 

0.96954 

C041901B 

0.89919 

T045303B 

0.86732 

C041901C 

0.91499 

T045303C 

0.87521 

C041901D 

0.89592 

T045303D 

0.86656 

T063001B 

0.90690 

T045303M 

0.97131 

T063001C 

0.91075 

T044000B 

0.86154 

T063001D 

0.91973 

T044000C 

0.87006 

T063001E 

0.93261 

T044000D 

0.84851 

T058301N 

0.86272 

T044000E 

0.88671 

T058301M 

0.95759 

T044000M 

0.87980 

NATLUNCH 

0.93597 

C034201B 

0.93716 

NATLUNCL 

0.92837 

C034201C 

0.95747 

REM  READ 

0.94925 

C034201M 

0.92975 

REMREADL 

0.94611 

C034402N 

0.90187 

REM  MATH 

0.94516 

C034402M 

0.97077 

REMMATHL 

0.94199 

O 

ERIC 


Proportion  of 
Variance 

0.93663 

0.94041 

0.94995 

0.91462 

0.95261 

0.91353 


693 


Table  C-12 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  12 


Contrast 

Proportion  of 
Variance 

Contrast 

FEMALE 

0.92889 

STGRDl-2 

BLACK 

0.92915 

STGRD3-5 

HISPANIC 

0.97024 

STGRD6> 

ASIAN 

0.92696 

SCHCHG-1 

MEXICAN 

0.94406 

SCHCHG-2 

PUER  RIC 

0.96474 

SCHCHG-3 

CUBN,OTH 

0.93574 

DIS@HOM2 

HISP-? 

0.96206 

DIS@HOM3 

NON  MSA 

0.92470 

1 DIS@HOM4 

MID  CTY5 

0.94452 

PGS>5 

FR/BTWN5 

0.94650 

PGS>10 

SML  TWN5 

0.95100 

ACADEMIC 

RURAL 5 

0.95572 

VOC/TECH 

URBAN  FR 

0.91208 

OTHERPGM 

MED  CITY 

0.87491 

HS  PGM-? 

SM  PLACE 

0.92893 

SEMENG-^ 

HS  GRAD 

0.94287 

#ENG-LIN 

POST  HS 

0.93664 

SEMMAT02 

COL  GRAD 

0.93811 

SEMMAT03 

PARED-? 

0.94441 

SEMMAT04 

S EAST 

0.89733 

SEMMAT-? 

CENTRAL 

0.91506 

#MAT-LIN 

WEST 

0.90241 

SEMSCI-^ 

PRIVATE 

0.92047 

#SCI-LIN 

CATHOLIC 

0.91554 

SEMHIS-^ 

lEP-NO 

0.91519 

#HIS-LIN 

LEP-NO 

0.83424 

SEMFLG-^ 

CHAPl-N 

0.79997 

#FLG-LIN 

RED  PRIC 

0.96378 

SEMVOC-^ 

FREE 

0.78963 

#VOC-LIN 

INFO  N/A 

0.82443 

SEMART-^ 

SCH/REF 

0.84379 

#ART-LIN 

HL-SOME 

0.86310 

G/R  22 

HL-ALWAY 

0.80564 

G/R  23 

HL-? 

0.88175 

G/R  24 

B008901N 

0.93293 

G/T  22 

B008901M 

0.77599 

G/T  23 

TVLIN-0 

0.97819 

G/T  24 

TV- QUAD 

0.97739 

G/T  25 

HW-NO 

0.96859 

G/P  22 

HW-YES 

0.97042 

G/P  23 

HWLIN-0 

0.92895 

G/P  24 

HWQUAD-0 

0.89826 

G/P  25 

HITEM=3 

0.89943 

G/S  22 

HITEM=4 

0.87168 

G/S  23 

MOMHOM-N 

0.81991 

R/T  24 

MOMHOM-? 

0.82492 

R/T  25 

DADHOM-N 

0.80223 

R/T  31 

DADHOM-? 

0.81180 

R/T  32 

MISS-2< 

0.95102 

R/T  3 3 

USA  >5 

0.85200 

R/T  3 4 

USA  3-5 

0.87304 

R/T  3 5 

USA  <3 

0.88979 

R/T  41 

USA-? 

0.88018 

R/T  42 

SNRM-LIN 

0.84637 

R/T  43 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.92029 

R/T  44 

0.92305 

0.94783 

R/T  45 

0.95298 

0.95254 

R/P  24 

0.88973 

0.90937 

R/P  25 

0.89812 

0.94307 

R/P  31 

0.90529 

0.86335 

R/P  32 

0 . 84951 

0.89647 

R/P  33 

0.88889 

0.91935 

R/P  34 

0.89844 

0.86792 

R/P  35 

0.90878 

0.83637 

R/P  41 

0 . 85378 

0.83038 

R/P  42 

0.93732 

0.71063 

R/P  43 

0.95245 

0.91092 

R/P  44 

0.94376 

0.90838 

R/P  45 

0 . 87956 

0.87226 

R/S  31 

0.97236 

0.87039 

R/S  32 

0.95845 

0.87227 

R/S  33 

0.96573 

0.94877 

R/S  41 

0.93453 

0.93763 

R/S  42 

0.96601 

0.95305 

R/S  43 

0.96263 

0.94796 

T/P  25 

0.80631 

0.95304 

T/P  31 

0.78804 

0.88033 

T/P  32 

0.79802 

0.83029 

T/P  33 

0.87480 

0.87348 

T/P  34 

0.80810 

0.80868 

T/P  35 

0.78901 

0.87828 

T/P  41 

0.79959 

0.74314 

T/P  42 

0.87871 

0.86063 

T/P  43 

0.81009 

0.83484 

T/P  44 

0.79426 

0.88550 

T/P  45 

0.81519 

0.82482 

T/P  51 

0.83235 

0.91066 

T/P  52 

0.77231 

0.91334 

T/P  53 

0.78145 

0.94772 

T/P  54 

0.81725 

0.72438 

T/P  55 

0 . 85260 

0.73222 

T/S  32 

0.94803 

0.74637 

T/S  33 

0.91833 

0.74718 

T/S  41 

0.93893 

0.86244 

T/S  42 

0.92502 

0.90798 

T/S  43 

0.94205 

0.83393 

T/S  52 

0.93511 

0.88757 

P/S  32 

0.95344 

0.93286 

P/S  33 

0.93108 

0.91873 

P/S  41 

0.94648 

0 . 88537 

P/S  42 

0.92627 

0.86606 

P/S  43 

0.93095 

0.87988 

P/S  51 

0.91136 

0.90904 

P/S  52 

0.95316 

0.88724 

P/S  53 

0 . 95220 

0.89011 

G/S  22 

0.91882 

0.90835 

G/S  23 

0.92822 

0.90889 

G/S  24 

0.89374 

0.91443 

G/S  25 

0.91531 

0.91285 

R/S  24 

0.90850 

ERIC 
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Table  C-12  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  12 


Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

R/S 

25 

0 . 89901 

B009301C 

0.96313 

M812001D 

R/S 

31 

0.89188 

B009301D 

0.96473 

M812001M 

R/S 

32 

0.87660 

B009301E 

0.94429 

M812003B 

R/S 

33 

• 0.91231 

B009301M 

0.85486 

M812003C 

R/S 

34 

0. 90430 

B009401B 

0.84742 

M812003D 

R/S 

35 

0 . 90898 

B009401C 

0.91078 

M812003M 

R/S 

41 

0.86200 

B009401D 

0.89509 

M812101B 

R/S 

42 

0.96148 

B009401M 

0.86608 

M812101C 

R/S 

43 

0 . 95207 

M812701B 

0.94299 

M812101D 

R/S 

44 

0.94882 

M812701C 

0.93163 

M812101E 

R/S 

45 

0.93292 

M812701D 

0.80006 

M812101F 

P/S 

25 

0.82873 

M812701M 

0.90305 

M812101G 

P/S 

31 

0 . 83186 

M812702B 

0.91453 

M812101M 

P/S 

32 

0.82603 

M812702C 

0.93249 

MMOOIOIB 

P/S 

33 

0.86589 

M812701D 

0.88981 

MMOOIOIC 

p/s 

34 

0.82924 

M812701M 

0.89179 

MMOOIOID 

p/s 

35 

0 . 85286 

M812703B 

0.91990 

MMOOIOIM 

p/s 

41 

0.84079 

M812703C 

0.93357 

MM00201B 

p/s 

42 

0.84679 

M812703D 

0.89724 

MM00201C 

p/s 

43 

0.83543 

M812703M 

0.75234 

MM00201D 

p/s 

44 

0.84071 

M812705B 

0.96117 

MM00201M 

p/s 

45 

0.84846 

M812705C 

0.95786 

MM00301B 

p/s 

51 

0 . 84557 

M812705D 

0.94391 

MM00301C 

p/s 

52 

0.88887 

M812705M 

0.70579 

MM00301D 

p/s 

53 

0 . 89799 

M812706B 

0.94209 

MM00301M 

p/s 

54 

0.90936 

M812706C 

0.94233 

MM00401B 

p/s 

55 

0.82638 

M812706D 

0.83293 

MM00401C 

T/S 

25 

0.78320 

M812706M 

0.69415 

MM00401D 

T/S 

31 

0.78789 

M812707B 

0.91333 

MM00401M 

T/S 

32 

0.79978 

M812707C 

0.92045 

MM00501B 

T/S 

33 

0 . 85158 

M812707D 

0.86804 

MM00501C 

T/S 

34 

0.79937 

M812707M 

0.72257 

MM00501D 

T/S 

35 

0.78979 

M812708B 

0.91510 

MM00501M 

T/S 

41 

0.79607 

M812708C 

0.92602 

B009601B 

T/S 

42 

0.82956 

M812708D 

0.80625 

B009601C 

T/S 

43 

0.80830 

M812708M 

0.84040 

B009601D 

T/S 

44 

0 . 79344 

M812709B 

0.93844 

B009601M 

T/S 

45 

0.80760 

M812709C 

0.93740 

B009601B 

T/S 

51 

0.83959 

M812709D 

0.81655 

B009601C 

T/S 

52 

0.80603 

M812709M 

0.79137 

B009601D 

T/S 

53 

0 . 81159 

M812710B 

0.93004 

B009601M 

T/S 

54 

0.81583 

M812710C 

0.93262 

M812712B 

T/S 

55 

0.87592 

M812710D 

0.90046 

M812712C 

S/S 

23 

0.94322 

M812710M 

0.73863 

M812712D 

S/S 

24 

0.92931 

M812711B 

0.89780 

M812712M 

S/S 

25 

0.92106 

M812711C 

0.92079 

M812713B 

S/S 

31 

0.93316 

M812711D 

0.83461 

M812713C 

S/S 

32 

0.93324 

M812711M 

0.80236 

M812713D 

S/S 

33 

0.92511 

M811201N 

0.84007 

M812713M 

S/S 

34 

0.90509 

M811201M 

0.83676 

M812201N 

S/S 

35 

0.92779 

M812001B 

0.83237 

M812201M 

SAMP  S2 

0.84496 

M812001C 

0.87232 

M812301N 

/R 

23 

0.89288 

M812001D 

0.85506 

M812301M 

/R 

24 

0.91002 

M812001M 

0.90125 

M812401N 

/R 

31 

0.95448 

M812002B 

0 . 83705 

M812401M 

B009301B 

0 . 94509 

M812001C 

0.87673 

M812701B 

ERIC  ^‘-'1 


Proportion  of 
Variance 

0 . 82854 
0.92458 
0.86440 
0 . 85592 
0.85387 
0.91317 
0.92796 
0.92880 
0.95261 
0.94985 
0.94558 
0.93445 
0.72041 
0.94768 
0.93958 
0.93427 
0.84862 
0.95098 
0.94926 
0.93590 
0.88012 
0.95948 
0.96544 
0.95838 
0.89161 
0.94518 
0.95820 
0.94797 
0.88277 
0.94211 
0.94736 
0.92707 
0.86945 
0.96300 
0.82502 
0.82014 
0.90689 
0.96814 
0.94405 
0.81806 
0.88549 
0.94634 
0.94978 
0.93745 
0.76943 
0.97058 
0.98047 
0.97780 
0.83470 
0.92856 
0.75873 
0.90158 
0.88514 
0.75286 
0.76428 
0.89155 


Table  C-12  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  12 


Contrast 

Proportion  of 
Variance 

Contrast 

M812701C 

0.94357 

OTHER 

M812701D 

0.94946 

MISSING 

M812701E 

0.85459 

OFFICE  W 

M812701M 

0.89191 

TECHNIC 

M810701B 

0.88919 

PROT  SRV 

M810701C 

0.94654 

SALES 

M810701D 

0.93144 

OWNER 

M810701E 

0.82369 

SERV  WRK 

M810701M 

0.85934  ■ 

SKILL  T 

M810707B 

0.89511 

LABORER 

M810707C 

0.93724 

OPERATOR 

M810707D 

0.93885 

FARMER 

M810707E 

0.80012 

HOMEMAKE 

M810707M 

0.78312 

MANAGER 

M810709B 

0.97172 

MILITARY 

M810709C 

0.97283 

TEACHER 

M810709D 

0.98478 

PROF ES SI 

M810709E 

0.98017 

PROFESS2 

M810709M 

0.81868 

BOllAOlN 

M810708B 

0.96101 

OFFICE 

M810708C 

0.97105 

TECHNIC 

M810708D 

0.96817 

PROT  SRV 

M810708E 

0.95129 

SALES 

M810708M 

0.77785 

OWNER 

M810710B 

0.90819 

SERV  WRK 

M810710C 

0.90931 

SKILL  TR 

M810710D 

0.95848 

LABORER 

M810710E 

0.84153 

OPERATOR 

M810710M 

0.83574 

FARMER 

M810705B 

0.89777 

HOMEMAKE 

M810705C 

0.91620 

MANAGER 

M810705D 

0.93308 

MILITARY 

M810705E 

0.87146 

TEACHER 

M810705M 

0.82100 

PROFESSl 

M810706B 

0.93812 

PROFESS2 

M810706C 

0.94644 

B012A02N 

M810706D 

0.95249 

M810901N 

M810706E 

0.93335 

M810901M 

M810706M 

0.88532 

M811801B 

M810711B 

0.93470 

M811801C 

M810711C 

0.94493 

M811801D 

M810711D 

0.94052 

M811801E 

M810711E 

0.91279 

M811801M 

M810711M 

0.88385 

M811001B 

B008301D 

0.93996 

M811001C 

B008301M 

0.78877 

M811001D 

B009901B 

0.91112 

M811001M 

B009901C 

0.89614 

M811002B 

B009901D 

0.95457 

M811002C 

B009901E 

0.86926 

M811002D 

B009901F 

0.90848 

M811002M 

B009901M 

0.84227 

M811014B 

VOC  COLL 

0.95667 

M811014C 

2 YR  COL 

0.95115 

M811014D 

4 YR  COL 

0.90968 

M811014M 

MILITARY 

0.96617 

M811014B 

O 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.95012 

M811014C 

0.92952 

0.89138 

M811014D 

0.91739 

0.94234 

M811014M 

0.72346 

0.97251 

M811004B 

0.86588 

0.96321 

M811004C 

0.87451 

0.95678 

M811004D 

0.85508 

0.96696 

M811004M 

0.83351 

0.95346 

M811005B 

0.88832 

0.97118 

M811005C 

0.92215 

0.95498 

M811005D 

0.84635 

0.95170 

M811005M 

0.84901 

0.96317 

M811006B 

0.91509 

0.85064 

M811006C 

0.94481 

0.97064 

M811006D 

0.86813 

0 . 97445 

M811006M 

0.82563 

0.93819 

M811007B 

0.94606 

0.95860 

M811007C 

0.89898 

0.97452 

M811007D 

0.91453 

0.84863 

M811007M 

0.80535 

0.97488 

M811008B 

0.93516 

0.96419 

M811008C 

0.90089 

0.96755 

M811008D 

0.92676 

0.96678 

M811008M 

0.86758 

0.95490 

M811012B 

0.94521 

0.97190 

M811012C 

0.95911 

0.95445 

M811012D 

0.92003 

0.94934 

M811012M 

0.90186 

0.94201 

M811009B 

0.96115 

0.94369 

M811009C 

0.96577 

0.95674 

M811009D 

0.96424 

0.96598 

M811009M 

0.91124 

0.97771 

M811011B 

0.90893 

0.95096 

M811011C 

0 . 97182 

0.94708 

M811011D 

0 . 93567 

0.91694 

M811011M 

0.90579 

0.81736 

M812801N 

0.84653 

0.80953 

M812801M 

0.83905 

0.82090 

C035701N 

0.86686 

0.91589 

C035701M- 

0.93081 

0.97455 

C035702N 

0.89168 

0.93853 

C035702M 

0.97529 

0.91037 

C035703N 

0.87631 

0.77042 

C035703M 

0.93179 

0.87459 

C037201M 

0.94402 

0.93298 

C037202M 

0.92866 

0.81116 

C037204M 

0.89740 

0.75042 

C037205M 

0.89736 

0.95531 

C037206M 

0.90217 

0.96639 

C037301M 

0.96296 

0.95585 

C037302M 

0.96132 

0.83289 

C037304M 

0.90405 

0.95058 

C037305M 

0.91941 

0.95387 

C036601B 

0.89703 

0.93483 

C036601C 

0.91500 

0.80040 

C036601D 

0.95387 

0.92280 

C036601M 

0.97024 
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Table  C-12  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  12 


Contrast 


C032207B 

C032207N 

C032207M 

C032209B 

C032209N 

C032209M 

C032210B 

C032210N 

C032210M 

C032211B 

C032211N 

C032211M 

C037701B 

C037701C 

C037701D 

C037701M 

C037702B 

C037702C 

C037702D 

C037702M 

C037703B 

C037703C 

C037703D 

C037703M 

C037704B 

C037704C 

C037704D 

C037704M 

C037705B 

C037705C 

C037705D 

C037705M 

C032402B 

C032402C 

C032402D 

C032402M 

C032401B 

C032401C 

C032401D 

C032404B 

C032404C 

C032404D 

C032406B 

C032406C 

C032406D 

C032407B 

C032407C 

C032408B 

C032408C 

C032409B 

C032409C 

C032409D 

C032410B 

C032410C 

C032410D 

C032411B 


Proportion  of 
Variance 

0.92302 
0.94402 
0.92083 
0.91337 
0.93446 
0 . 94787 
0.93101 
0.94633 
0 . 93765 
0.88637 
0 . 90622 
0.94126 
0 . 90046 
0 . 89540 
0 . 89470 
0 . 95870 
0.91199 
0 . 92385 
0.89213 
0 . 96807 
■ 0.89290 
0 . 90844 
0.91385 
0 . 98568 
0.88396 
0 . 88768 
0.91019 
0 . 93188 
0 . 89396 
0 . 89019 
0 . 86953 
0 . 94772 
0.92107 
0 . 92528 
0.90811 
0.99466 
0 . 89473 
0.93144 
0 . 89418 
0.88648 
0 . 94295 
0 . 93133 
0.88220 
0.94367 
0 . 94071 
0.85721 
0.90257 
0 . 89199 
0 . 87317 
0.92022 
0.93469 
0 . 90807 
0.90849 
0.93557 
0.92060 
0.92589 


Contrast 


C032411C 

C032411D 

C032412B 

C032412C 

C032412D 

C032413B 

C032413C 

C032413D 

C032413M 

C032414B 

C032414C 

C032414D 

C032415B 

C032415C 

C032502B 

C032502C 

C032502D 

C032502M 

C032503B 

C032503C 

C032503M 

C032505B 

C032505C 

C032506B 

C032506C 

C032506D 

C033601B 

C033601C 

C033601D 

C033601M 

C036501B 

C036501C 

C037801B 

C037801C 

C037801D 

C037801E 

C037801F 

C037801G 

C037801M 

C038001B 

C038001C 

C038001D 

C038001M 

C038301N 

C038301M 

C038801N 

C038801M 

C034101M 

C034102M 

C034103M 

C034104M 

C034105M 

C034106M 

C034107M 

C034108M 

C034109M 


Proportion  of 
Variance 

0.93713 

0.90612 

0.91793 

0.94392 

0.90855 

0.90748 

0.94639 

0.94378 

0.97446 

0.91207 

0.94107 

0 . 88274 

0.92605 

0.92522 

0.90450 

0.90350 

0.94285 

0.97611 

0.90781 

0.90315 

0 . 97893 

0.88865 

0.89919 

0.89021 

0.88718 

0.92530 

0.91818 

0 . 92725 

0.88997 

0.96468 

0.89604 

0.90429 

0.89556 

0.90588 

0.91885 

0.91122 

0.90982 

0.91465 

0.98176 

0.84995 

0.90179 

0.93271 

0.98025 

0 . 88718 

0.93248 

0.90293 

0.92243 

0.88453 

0.88179 

0.88071 

0.90456 

0 . 89147 

0 . 87657 

0.90138 

0.89179 

0.89172 


Contrast 

C037203M 

C037306M 

C035002N 

C035002M 

C035003N 

C035003M 

C035006N 

C035006M 

C035007N 

C035007M 

C040201C 

C040201D 

C040201E 

C040201F 

C040201M 

C040202C 

C040202D 

C040202E 

C040203E 

C040203F 

C040204B 

C040204C 

C040204D 

C040204M 

C040301M 

C040302M 

C040303M 

C040304M 

C040305M 

C040306M 

C040307M 

C040308M 

C040309M 

C040310M 

C040311M 

C040401N 

C040401M 

C040402N 

C040402M 

C040403N 

C040404N 

C040501M 

C040502M 

C040503M 

C040504M 

C040505M 

C040506M 

C040601M 

C040602M 

C040603M 

C040604M 

C040605M 

C040701M 

C040702M 

C040703M 

C040704M 


7u3 


Proportion  of 
Variance 

0.88792 
0.95734 
0.93828 
0.98503 
0.92750 
0 . 97758 
0.90580 
0.96848 
0.87812 
0.93619 
0.92029 
0.95054 
0.95411 
0.92232 
0.98935 
0.91346 
0.95124 
0.92503 
0.92372 
0.94662 
0 . 89422 
0.88909 
0.88218 
0.95160 
0.92441 
0.87495 
0.88201 
0.91946 
0.87201 
0.88846 
0.86896 
0.86455 
0.89038 
0.89854 
0 . 90704 
0.95301 
0.98325 
0.92195 
0.94504 
0.95216 
0.94319 
0.89832 
0.85814 
0.87339 
0.88199 
0.88898 
0.87669 
0.89355 
0 . 89762 
0.88420 
0 . 87274 
0 . 85977 
0.90518 
0.89943 
0.90987 
0.87241 


Table  C-12  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Mathematics  Main  Conditioning  Variables,  Grade  12 


Proportion  of  Proportion  of 

Contrast Variance Contrast Variance Contrast 


C040705M 

0 . 91477 

C040801B 

0.89997 

C040801C 

0.88676 

C040801D 

0.90481 

C040801E 

0.89002 

C040801F 

0.90652 

C040801G 

0.91176 

C040801M 

0.91525 

C040802B 

0.90037 

C040802C 

0.90687 

C040802D 

0.91291 

C040802E 

0.88286 

C040802F 

0.90444 

C040802G 

0.90068 

C040802M 

0.92939 

C040803B 

0.87355 

C040803C 

0 . 87565 

C040803D 

0 . 87617 

C040803G 

0.90965 

C040803M 

0.88149 

C040804B 

0.86226 

C040804C 

0 . 91401 

C040804D 

0.91207 

C040804E 

0.89213 

C040804F 

0.87585 

C040804G 

0.92287 

C040804M 

0.94197 

C040901M 

0.91375 

C040902M 

0.89865 

C040903M 

0.91475 

C040904M 

0.90002 

C040905M 

0.93204 

C041001B 

0.91548 

C041001C 

0 . 87835 

C041001D 

0 . 88912 

C041101B 

0.91241 

C041101C 

0 . 89830 

C036001B  • 

0.89703 

C036001C 

0.88469 

C036001D 

0.87517 

NATLUNCH 

0.89506 

NATLUNCL 

0 . 89650 

REM  READ 

0.87890 

REMREADL 

0.90861 

REMMATHL 

0.90472 

Proportion  of 
Variance 
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Table  C- 13 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

FEMALE 

0.91651 

STGRDl-2 

BLACK 

0.93312 

STGRD3> 

HISPANIC 

0.96527 

SNRM-LIN 

ASIAN 

0.95176 

B009201B 

MEXICAN 

0.93869 

B009201C 

PUER  RIC 

0.92524 

B009201D 

CUBN, OTH 

0.96503 

B009201M 

HISP-? 

0.81924 

DIS0HOM2 

NON  MSA 

0.95741 

DIS@HOM3 

MID  CTY5 

0.95016 

DIS@HOM4 

FR/BTWN5 

0.94663 

1-2  WEEK 

SML  TWN5 

0.95003 

1-2  MTH 

RURAL5 

0.96741 

NEVER 

URBAN  FR 

0.90792 

NO  COMP 

MED  CITY 

0.91599 

MISSING 

SM  PLACE 

0.93846 

G/R  22 

HS  GRAD 

0.93658 

G/R  23 

POST  HS 

0.92025 

G/R  24 

COL  GRAD 

0.92303 

G/T  22 

PARED- ? 

0.93244 

G/T  23 

S EAST 

0.89914 

G/T  24 

CENTRAL 

0.91367 

G/T  25 

WEST 

0.89828 

G/P  22 

PRIVATE 

0.92836 

G/P  23 

CATHOLIC 

0.92022 

G/P  24 

lEP-NO 

0.93526 

G/P  25 

LEP-NO 

0.79947 

G/S  22 

CHAPl-N 

0.75390 

G/S  23 

REDUCED 

0.91147 

R/T  24 

FREE 

0.72822 

R/T  25 

INFO  NA 

0.84735 

R/T  31 

SCH  REF 

0.89453 

R/T  32 

HL-SOME 

0.84839 

R/T  33 

HL-ALWAY 

0.79850 

R/T  34 

HL-? 

0.73669 

R/T  35 

TVLIN-0 

0.98029 

R/T  41 

TV-QUAD 

0.98000 

R/T  42 

HW-NO 

0.96073 

R/T  43 

HW-YES 

0.96470 

R/T  44 

HWLIN-0 

0.98137 

R/T  45 

HWQUAD-0 

0.97281 

R/P  24 

1-2  HRS 

0.91902 

R/P  25 

3-4  HRS 

0.94019 

R/P  31 

5-6  HRS 

0.95904 

R/P  32 

7-8  HRS 

0.96322 

R/P  33 

9-10  HRS 

0.96691 

R/P  34 

> 10  HRS 

0.94955 

R/P  35 

B009101M 

0.77222 

R/P  41 

HITEM=3 

0.86009 

R/P  42 

HITEM=4 

0.81060 

R/P  43 

MISS-2< 

0.92655 

R/P  44 

USA  >5 

0.90593 

R/P  45 

USA  3-5 

0.92725 

R/S  31 

USA  <3 

0.92754 

R/S  32 

USA-? 

0.78635 

R/S  33 

o 

ERIC 

. • V 

Proportion  of  Proportion  of 

Variance Contrast  Variance 


0.87161 

R/S  41 

0.94664 

0.84855 

R/S  42 

0.96346 

0.89875 

R/S  43 

0.97550 

0.85946 

T/P  25 

0.78665 

0.89060 

T/P  31 

0.83830 

0.84523 

T/P  32 

0.78928 

0.81101 

T/P  33 

0.76732 

0.86885 

T/P  34 

0.76387 

0.91087 

T/P  35 

0.78267 

0.78205 

T/P  41 

0.76155 

0.94751 

T/P  42 

0.75914 

0.95215 

T/P  43 

0.79689 

0.94409 

T/P  44 

0.80321 

0.93048 

T/P  45 

0.83598 

0.86738 

T/P  51 

0.81841 

0.91396 

T/P  52 

0.79478 

0.90912 

T/P  53 

0.81808 

0.96268 

T/P  54 

0.79771 

0.69418 

T/P  55 

0.80328 

0.66997 

T/S  32 

0.95828 

0.77415 

T/S  33 

0.93286 

0.73587 

T/S  41 

0.94266 

0.91395 

T/S  42 

0.93954 

0.88694 

T/S  43 

0.95438 

0.84063 

T/S  51 

0.94557 

0.84875 

T/S  52 

0.97040 

0.92396 

T/S  53 

0.96619 

0.90171 

P/S  32 

0.95468 

0.86023 

P/S  33 

0.92734 

0.89305 

P/S  41 

0.94834 

0.87802 

P/S  42 

0.90951 

0.89389 

P/S  43 

0.94043 

0.86112 

P/S  51 

0.92263 

0.86129 

P/S  52 

0.94462 

0.88236 

P/S  53 

0.92977 

0.88877 

B008901N 

0.95010 

0.91771 

B008901M 

0.64425 

0.92809 

SAFE 

0.87260 

0.93044 

UNSAFE 

0.90710 

0.95100 

VRUNSAFE 

0.88994 

0.89904 

MISSING 

0.84004 

0.88076 

MOMHOM-N 

0.80557 

0.90960 

MOMHOM-? 

0.79943 

0.91029 

DADHOM-N 

0.83554 

0.89805 

DADHOM- ? 

0.82552 

0.89852 

K811001B 

0.82197 

0.89951 

K811001C 

0.85117 

0.89486 

K811001M 

0.78778 

0.97268 

K811002B 

0.83114 

0.96015 

K811002C 

0.84120 

0.96464 

K811002M 

0.74258 

0.96335 

K811003B 

0.80011 

0.95642 

K811003C 

0.79434 

0.96868 

K811003M 

0.64436 

0.96432 

K811004B 

0.80713 

Table  C-13  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

K811004C 

0.80115 

K811605M 

K811004M 

0.67661 

K811606B 

K811005B 

0.88234 

K811606C 

K811005C 

0.86102 

K811606D 

K811005M 

0.72124 

K811606M 

K811006B 

0.94116 

K811607B 

K811006C 

0.92431 

K811607C 

K811006M 

0.77614 

K811607D 

K811007B 

0.88338 

K811607M 

K811007C 

0.87999 

K811608B 

K811007M 

0.74157 

K811608C 

K811008B 

0.84335 

K811608D 

K811008C 

0.83097 

K811608M 

K811008M 

0.83133 

K811609B 

K811101M 

0.85314 

K811609C 

K811102M 

0.75917 

K811609D 

K811103M 

0.74925 

K811609M 

K811104M 

0.76326 

K811610B 

K811105M 

0.74473 

K811610C 

K811106M 

0.70566 

K811610D 

K811107M 

0.74751 

K811610M 

K811108M 

0.92666 

K811611B 

K811201B 

0 . 88957 

K811611C 

K811201C 

0.91390 

K811611D 

K811201D 

0.92028 

K811611M 

K811201E 

0.93299 

K811612B 

K811201M 

0 . 87365 

K811612C 

K811301B 

0.92638 

K811612D 

K811301C 

0.93837 

K811612M 

K811301D 

0.94938 

K811701B 

K811301E 

0 . 96340 

K811701C 

K811301F 

0.96269 

K811701D 

K811301M 

0.88178 

K811701M 

K811401N 

0 . 84490 

K811702B 

K811401M 

0.89888 

K811702C 

K811501N 

0.75563 

K811702D 

K811501M 

0.90556 

K811702M 

K811601B 

0.90579 

K811703B 

K811601C 

0.93331 

K811703C 

K811601D 

0.86144 

K811703D 

K811601M 

0.78641 

K811703M 

K811602B 

0.93446 

K811704B 

K811602C 

0.93522 

K811704C 

K811602D 

0 . 91488 

K811704D 

K811602M 

0 . 80978 

K811704M 

K811603B 

0.94079 

K811705B 

K811603C 

0.94416 

K811705C 

K811603D 

0.91219 

K811705D 

K811603M 

0.77565 

K811705M 

K811604B 

0.93590 

K811801B 

K811604C 

0.91399 

K811801C 

K811604D 

0.91171 

K811801M 

K811604M 

0.76089 

K811901B 

K811605B 

0.94002 

K811901C 

K811605C 

0.94156 

K811901M 

K811605D 

0.92541 

SMOOIOIB 

O 


Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.81252 

SMOOIOIC 

0.85860 

0 . 92744 

SMOOIOID 

0.90776 

0.93016 

SMOOIOIM 

0.86142 

0.90992 

SM00201B 

0.90458 

0.73070 

SM00201C 

0.90363 

0.91407 

SM00201D 

0.90657 

0 . 91450 

SM00201M 

0.88572 

0 . 90192 

SM00301B 

0.87162 

0.77303 

SM00301C 

0.85440 

0 . 91591 

SM00301D 

0 . 90224 

0.92996 

SM00301M 

0.83821 

0.90372 

SM00401B 

0 . 88524 

0 . 82830 

SM00401C 

0.89177 

0.89975 

SM00401D 

0.92217 

0.93252 

SM00401M 

0.90223 

0.88360 

SM00501B 

0 . 85850 

0 . 83978 

SM00501C 

0.87820 

0.93662 

SM00501D 

0.86234 

0.92396 

SM00501M 

0 . 88279 

0.92722 

B009501N 

0.93679 

0.80885 

B009501C 

0.81293 

0.94193 

B009501M 

0.85361 

0.93725 

B009502N 

0 . 95137 

0.93167 

B009502C 

0 . 84209 

0.83030 

B009502M 

0 . 85508 

0.94534 

C030901B 

0.89403 

0.93793 

C030901C 

0 . 90221 

0.91572 

C030901M 

0.97962 

0.86702 

C037101B 

0.92384 

0.90475 

C037101C 

0.92028 

0.91033 

C037101N 

0 . 94104 

0.84629 

C037101M 

0 . 94257 

0.87126 

C031212B 

0 . 87125 

0.94339 

C031212M 

0.96859 

0.94654 

C031205B 

0 . 89767 

0.94051 

C031205C 

0 . 90190 

0.88966 

C031205M 

0 . 94785 

0.93095 

C031213M 

0.95871 

0.94496 

C031214B 

0.92465 

0.92911 

C031214C 

0.93150 

0.87002 

C031214D 

0 . 89904 

0.89118 

C031214E 

0.91092 

0.93560 

C031214M 

0.94845 

0.89198 

C031603N 

0.89974 

0.91996 

C031603M 

0.94317 

0 . 93027 

C031607N 

0.89448 

0.93831 

C031607M 

,0.96400 

0 . 90084 

C031601N 

0.87885 

0 . 92413 

C031601M 

0.96929 

0.92272 

C031610N 

0.91936 

0.90026 

C031610M 

0.96212 

0.89963 

C031606N 

0.90036 

0.91691 

C031606M 

0.95365 

0.90711 

C035701N 

0.89605 

0.88535 

C035701M 

0.91746 

0.85426 

C035702N 

0.90389 
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Table  C-13  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for- 
Science  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

C035702M 

0.93199 

C037704C 

0.91613 

C036501C 

0.93032 

C035703N 

0.88916 

C037704D 

0.90235 

C036501M 

0.97935 

C035703M 

0.93495 

C037704M 

0.94786 

C037801B 

0.91450 

C037201M 

0.95553 

C037705B 

0.92164 

C037801C 

0.89298 

C037202M 

0.92137 

C037705C 

0.90676 

C037801D 

0.92829 

C037207M 

0.94860 

C037705D 

0.89117 

C037801E 

0.91023 

C037204M 

0.90422 

C032402B 

0.9.0070 

C037801F 

0.90574 

C037205M 

0.89754 

C032402C 

0.94641 

C037801G 

0.90165 

C037206M 

0.93927 

C032402D 

0.94378 

C037801H 

0.90540 

C037301M 

0.96994 

C032402M 

0.95157 

C037801M 

0 . 93753 

C037302M 

0.95988 

C032401B 

0.93303 

C037901B 

0 . 90514 

C037303M 

0.94894 

C032401C 

0.94412 

C037901C 

0.90170 

C037304M 

0.90594 

C032401D 

0.94804 

C037901D 

0.90960 

C037305M 

0.95441 

C032401M 

0.98947 

C038001B 

0 . 89636 

C037401M 

0.89597 

C032404B 

0.92340 

C038001C 

0.90399 

C037402M 

0.90113 

C032404C 

0.95081 

C038001D 

0.90048 

C037403M 

0.92514 

C032404D 

0.94137 

C038001F 

0 . 88251 

C037404M 

0.89399 

C032404M 

0.96651 

C038301N 

0.90020 

C037405M 

0.91465 

C032406B 

0.91596 

C038801N 

0 . 90155 

C037406M 

0.87946 

C032406C 

0.90435 

C038801M 

0 . 97431 

C037501M 

0.90453 

C032407B 

0.91061 

C034101M 

0.92230 

C037502M 

0.89548 

C032407C 

0.95076 

C034102M 

0.89611 

C037503M 

0.90802 

C032407D 

0.95552 

C034103M 

0 . 94551 

C037504M 

0.92563 

C032407M 

0.98151 

C034104M 

0 . 91301 

C037505M 

0.92290 

C032408B 

0.90945 

C034105M 

0.89582 

C037601M 

0.93328 

C032408C 

0.94772 

C034106M 

0 . 91470 

C037602M 

0.92709 

C032408D 

0.94886 

C034107M 

0.90153 

C037603M 

0.95951 

C032408M 

0.96953 

C034108M 

0.90872 

C037604M 

0.92443 

C032409B 

0.93628 

C034109M 

0 . 90109 

C037605M 

0.95223 

. C032409C 

0.93266 

T055901B 

0 . 87479 

C036601B 

0.88504 

C032409D 

0.94444 

T055901M 

0 . 96866 

C036601C 

0.93206 

C032410C 

0.93272 

T056001B 

0 . 85043 

C036601D 

0.95477 

C032411B 

0.90528 

T056001C 

0 . 86327 

C036601M 

0.95288 

C032411C 

0.92930 

T056001D 

0 . 86537 

C032207B 

0.92922 

C032412C 

0.93921 

T056001E 

0 . 90289 

C032207N 

0.91089 

C032413B 

0.88499 

T056001F 

0 . 86833 

C032207M 

0.93632 

C032413C 

0.89413 

T056001M 

0 . 94320 

C032209B 

0.89828 

C032414B 

0.92372 

T040301B 

0 . 90532 

C032209N 

0.87527 

C032414C 

0.91823 

T040301C 

0 . 93146 

C032209M 

0.98238 

C032415B 

0.93497 

T040301D 

0 . 93576 

C032210B 

0.90543 

C032415C 

0.90676 

T040301E 

0 . 94781 

C032210N 

0.91405 

C032502B 

0.90056 

T040301M 

0 . 98027 

C032210M 

0.94207 

C032502C 

0.89916 

T056101B 

0 .91259 

C032211B 

0.89516 

C032502D 

0.93295 

T056101C 

0 . 93748 

C032211N 

0.90357 

C032503B 

0.91273 

T056101D 

0.94770 

C032211M 

0.99172 

C032503C 

0.93271 

T056101E 

0 . 96263 

C037701B 

0.91761 

C032505B 

0.90764 

T056101M 

0 . 96167 

C037701C 

0.90801 

C032505C 

0.90822 

T056102B 

0 . 95393 

C037701D 

0.92439 

C032505D 

0.91176 

T056102C 

0.93711 

C037702B 

0.91405 

C032506B 

0.89234 

T056102D 

0.94738 

C037702C 

0.93501 

. C032506C 

0.91856 

T056102E 

0 . 95662 

C037702D 

0.93940 

C033601B 

0.91617 

T056102M 

0 . 94851 

C037703B 

0.93418 

C033601C 

0.90915 

T056201B 

0 .88672 

C037703C 

0.94161 

C033601D 

0.91403 

T056201C 

0 . 87479 

C037703D 

0.94797 

C033601M 

0.95663 

T056201D 

. 0.88203 

C037704B 

0.92282 

C036501B 

0.91852 

T056201E 

0.84446 

7u7 


Table  C-13  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

T056201F 

0.85825 

T056403M 

T056201M 

0.91983 

T056404M 

T040501N 

0.83560 

T056405M 

T040501C 

0.90736 

T056413M 

T040501M 

0.89047 

T056414M 

T040506N 

0 . 94737 

T056415M 

T040506C 

. 0.94707 

T056416M 

T040506M 

0 . 94102 

T056406M 

T040504N 

0.94207 

T056407M 

T040504C 

0 . 92334 

T056408M 

T040504M 

0.95367 

T056409M 

T040507N 

0 . 94296 

T056410M 

T040507C 

0.94279 

T056411M 

T040507M 

0 . 94702 

T056412M 

T040508N 

0 . 94706 

T056501B 

T040508C 

0.93476 

T056501C 

T040508M 

0 . 95486 

T056501D 

T040505N 

0.88670 

T056501E 

T040505C 

0 . 87901 

T056501M 

T040505M 

0.88696 

T058101B 

T056301B 

0 . 86576 

T058101C 

T056301C 

0.93282 

T058101D 

T056301D 

0.93167 

T058101E 

T056301E 

0.88357 

T058101M 

T056301G 

0.89492 

T056601B 

T056301M 

0 . 95311 

T056601C 

T040701M 

0.87840 

T056601D 

T040706M 

0 . 86770 

T056601E 

T040707M 

0 . 86427 

T056601M 

T040703M 

0.89060 

T058201B 

T040704M 

0.89093 

T058201C 

T040710M 

0.88417 

T058201D 

T040711M 

0.92342 

T058201E 

T040712M 

0.91839 

T058201M 

T040713M 

0.93532 

T056701M 

T040708M 

0.86960 

T056702M 

T040709M 

0.88466 

T056703M 

T040705M 

0.88844 

T056704M 

T040801M 

0.88669 

T056705M 

T040807M 

0.86361 

T056706M 

T040808M 

0.86571 

T056707M 

T040803M 

0.89119 

T056708M 

T040804M 

0.87594 

T056709M 

T040814M 

0.87928 

T056710M 

T040815M 

0.95483 

T056711M 

T040816M 

0.94150 

T056712M 

T040817M 

0.91605 

T056713M 

T040809M 

0.86332 

T041201B 

T040810M 

0.88005 

T041201C 

T040811M 

0.87776 

T041201D 

T040812M 

0.87941 

T041201M 

T040813M 

0.86773 

T041302N 

T040805M 

0.86343 

T041302M 

T040806M 

0.87073 

T041303N 

T056401M 

0.88847 

T041303M 

T056402M 

0.87790 

T056801B 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.85589 

T056801C 

0 . 89230 

0.86367 

T056801D 

0 . 92794 

0.87848 

T056801E 

0.89607 

0.86716 

T056801F 

0 . 88949 

0.90389 

T056801M 

0.94958 

0.90062 

T060201B 

0.92000 

0.90838 

T060201C 

0.94579 

0.87295 

T060201D 

0.92003 

0.87562 

T060201E 

0.91478 

0.87219 

T060201F 

0.90732 

0.86749 

T060201M 

0.95858 

0.87411 

T060301M 

0.87421 

0.90319 

T060311M 

0.88509 

0.89205 

T060321M 

0.87497 

0.88647 

T060331M 

0.88300 

0.88938 

T060341M 

0.86623 

0.89411 

T060302M 

0 . 90604 

0.89143 

T060312M 

0.85498 

0.94306 

T060322M 

0.91112 

0.88634 

T060332M 

0.88119 

0.89152 

T060342M 

0.88255 

0.87735 

T060303M 

0.88005 

0.88157 

T060313M 

0.83306 

0.96791 

T060323M 

0.92645 

0.88791 

T060333M 

0.88680 

0.87160 

T060343M 

0.88312 

0.89490 

T060304M 

0.89420 

0.89456 

T060314M 

0.86209 

0.96538 

T060324M 

0.93375 

0 . 88404 

T060334M 

0.88551 

0.85907 

T060344M 

0.89704 

0.88962 

T060305M 

0.92229 

0.86467 

T060315M 

0.88554 

0 . 96544 

T060325M 

0.90188 

0.86285 

T060335M 

0.89047 

0 . 87917 

T060345M 

0.87521 

0.85590 

T060306M 

0.87953 

0 . 85823 

T060316M 

0.85377 

0.84873 

T060326M 

0.88992 

0.85463 

T060336M 

0.84977 

0.85042 

T060346M 

0 . 85970 

0 . 86160 

T060307M 

0 . 88846 

0 . 86150 

T060317M 

0 . 96011 

0.86055 

T060327M 

0 . 95994 

0 . 87227 

T060337M 

0.90362 

0 . 86345 

T060347M 

0.90776 

0.85920 

T060401M 

0 . 84814 

0.90893 

T060402M 

0 . 86197 

0.92565 

T060403M 

0 . 86825 

0 . 88467 

T060404M 

0 . 86779 

0.98547 

T060405M 

0.86266 

0.93022 

T060501N 

0 . 85967 

0.90144 

T060501M 

0 . 93416 

0.91305 

T060601B 

0 . 87529 

0.91998 

T060601C 

0.88302 

0.86554 

T060601D 

0 . 89945 
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Table  C-13  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  4 


Contrast 

Proportion  of 
Variance 

Contrast 

T060601M 

0.98632 

T060801B 

T060602B 

0 . 89151 

T060801C 

T060602C 

0 . 93844 

T060801M 

T060602D 

0.90940 

T060901B 

T060602M 

0.94668 

T060901C 

T060603B 

0 . 90724 

T061001N 

T060603C 

0.93143 

T061001M 

T060603D 

0 . 90517 

T061101B 

T060603M 

0 . 98055 

T061101C 

T060604B 

0.89483 

T061101M 

T060604C 

0.92361 

T061102B 

T060604D 

0.88300 

T061102C 

T060604M 

0 . 94484 

T061102M 

T060605B 

0.88991 

T061103B 

T060605C 

0 . 92415 

T061103C 

T060605D 

0.93407 

T061103M 

T060605M 

0.98883 

T061104B 

T060606B 

0.87899 

T061104C 

T060606C 

0 .94199 

T061104M 

T060606D 

0.94163 

T061105B 

T060606M 

0.97172 

T061105C 

T060607B 

0 . 91855 

T061105M 

T060607C 

0.91617 

T061106B 

T060607D 

0.86690 

T061106C 

T060607M 

0.96799 

T061107B 

T060608B 

0.91945 

T061107C 

T060608C 

0.92808 

T061107M 

T060608D 

0.88940 

T061108B 

T060608M 

0.97707 

T061108C 

T060609B 

0.91789 

T061108M 

T060609C 

0.95337 

T061109B 

T060609D 

0.90651 

T061109C 

T060609M 

0.98558 

T061109M 

T060610B 

0.89244 

T061201N 

T060610C 

0.93467 

T061201M 

T060610D 

0.90222 

T061301B 

T060610M 

0.96781 

T061301C 

T060611B 

0.88331 

T061301D 

T060611C 

0.90780 

T061301E 

T060611D 

0.93529 

T061301M 

T060611M 

0.97020 

T061302B 

T060701B 

0.88527 

T061302C 

T060701C 

0.86155 

T061302D 

T060701D 

0.89594 

T061302E 

T060702B 

0.92325 

T061302M 

T060702C 

0.93876 

T061303B 

T060702D 

0.88673 

T061303C 

T060703B 

0.89777 

T061303D 

T060703C 

0.95750 

T061303E 

T060703D 

0.92713 

T061304B 

T060704B 

0 . 89563 

T061304C 

T060704C 

0.92015 

T061304D 

T060704D 

0.95093 

T061304E 

T060705B 

0.88473 

T061305B 

T060705C 

0 . 92378 

T061305C 

T060705D 

0.94478 

T061305D 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

0.92551 

T061305E 

0.94363 

0.93240 

T061306B 

0.88254 

0.98743 

T061306C 

0.90312 

0.93158 

T061306D 

0.90891 

0.93591 

T061306E 

0.93034 

0.91202 

T061307B 

0.88521 

0.98196 

T061307C 

0.86756 

0.88422 

T061307D 

0.90171 

0.86186 

T061307E 

0.91337 

0.97686 

T061307M 

0.96368 

0.85567 

T061308B 

0.89370 

0.96220 

T061308C 

0.88299 

0.98334 

T061308D 

0.87586 

0.87809 

T061308E 

0.91111 

0.88451 

T061308M 

0.97867 

0.97978 

T061309B 

0.88476 

0.89582 

T061309C 

0 . 88316 

0.89609 

T061309D 

0.87785 

0.97552 

T061309E 

0 . 89188 

0.89677 

T061309M 

0.97721 

0.90152 

T061310B 

0 . 88162 

0.95800 

T061310C 

0 . 85340 

0.91773 

T061310D 

0.86935 

0.91940 

T061310E 

0 . 89975 

0.85928 

T061310M 

0 . 97042 

0.88080 

T061401B 

0.89801 

0.95604 

T061401C 

0 . 91559 

0.90244 

T061401D 

0 . 89104 

0.91920 

T061401M 

0 . 97890 

0.96316 

T061501B 

0 . 90299 

0.90216 

T061501C 

0 . 90394 

0.91645 

T061501D 

0 . 88877 

0.96285 

T061501E 

0 . 88255 

0.85080 

T061501F 

0.88173 

0.96119 

T061501M 

0 . 88715 

0.91510 

T061601M 

0 . 88212 

0.90641 

T061611M 

0 . 87828 

0.88445 

T061621M 

0 . 86907 

0,90907 

T061631M 

0 . 86113 

0.97112 

T061641M 

0.87562 

0.88943 

T061651M 

0 . 89760 

0.89580 

T061701C 

0 . 88626 

0 . 86364 

T061701E 

0 . 88636 

0.86412 

T061701M 

0 . 94247 

0.97288 

T061801B 

0 . 87449 

0.90705 

T061801C 

0.87469 

0.92973 

T061801D 

0.85703 

0.91196 

T061801E 

.0.88894 

0.87885 

T061801M 

0 . 84784 

0.89552 

T061901M 

0.95569 

0.91273 

T062001B 

0.88898 

0.90015 

T062001C 

0.85320 

0.90184 

T062001M 

0.94992 

0.89481 

T062002B 

0.89213 

0 . 90307 

T062002C 

0.88992 

0.88751 

T062002M 

0.95118 

Table  C-13  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  4 


Contrast 


T062003B 

T062003C 

T062003D 

T062003M 

T062101B 

T062101C 

T062101D 

T062101M 

T062201M 

T062202M 

T062203M 

T062204M 

T062205M 

T062206M 

T062207M 

T062208M 

T062209M 

T062210M 

T062301B 

T062301C 

T062301D 

T062301E 

T062401B 

T062401C 

T062401D 

T062401M 

NATLUNCH 

NATLUNCL 

REMREADL 

REMMATHL 

NATLUN4L 

REMREA4L 

REMMAT4L 


Proportion  of 
Variance 

0.91950 

0.89596 

0.90014 

0.93908 

0.87928 

0.87528 

0.89825 

0.95016 

0.86708 

0.86001 

0.84448 

0.86846 

0.85517 

0.86636 

0.86048 

0.86099 

0.86118 

0.84056 

0.91500 

0.89514 

0.89244 

0.86790 

0.86494 

0.87876 

0.85596 

0.90603 

0.88877 

0.94512 

0.94139 

0.94325 

0.94128 

0.92554 

0.91214 


Contrast 


Proportion  of 

Variance  Contrast 


Proportion  of 
Variance 
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Table  C-14 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  8 


Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

FEMALE 

0.91614 

BOIOIOIC 

0.93096 

R/P 

34 

BLACK 

.0.92990 

BOIOIOID 

0.91930 

R/P 

35 

HISPANIC 

0.96684 

BOIOIOIE 

0.92367 

R/P 

41 

ASIAN 

0.94591 

BOIOIOIF 

0.93928 

R/P 

42 

MEXICAN 

0.92894 

BOIOIOIG 

0.95512 

R/P 

43 

PUER  RIC 

0.93935 

BOIOIOIH 

0.93420 

R/P 

44 

CUBN.OTH 

0 . 93707 

BOIOIOIM 

0 . 87041 

R/P 

45 

HISP-? 

0.91996 

USA  >5 

0.89993 

R/S 

31 

NON  MSA 

0.94360 

USA  3-5 

0.86697 

R/S 

32 

MID  CTY5 

0 . 93750 

USA  <3 

0.88452 

R/S 

33 

FR/BTWN5 

0.95233 

USA-? 

0.79048 

R/S 

41 

SML  TWN5 

0.96189 

SNRM-LIN 

0.90986 

R/S 

42 

RURAL 5 

0.96124 

DIS0HOM2 

0.89041 

R/S 

43 

URBAN  FR 

0.92243 

DIS0HOM3 

0.92742 

T/P 

25 

MED  CITY 

0.91865 

DIS0HOM4 

0.81911 

T/P 

31 

SM  PLACE 

0.94745 

1-2  WEEK 

0.93788 

T/P 

32 

HS  GRAD 

0 . 94717 

1-2  MTH 

0.95331 

T/P 

33 

POST  HS 

0.93099 

NEVER 

0.94437 

T/P 

34 

COL  GRAD 

0.93104 

NO  COMP 

0.92631 

T/P 

35 

PARED-? 

0.94345 

MISSING 

0.85984 

T/P 

41 

S EAST 

0.92674 

LIFESCI 

0.92125 

T/P 

42 

CENTRAL 

0.91034 

PHYSSCI 

0.93355 

T/P 

43 

WEST 

0.92103 

EATHSCI 

0.92607 

T/P 

44 

PRIVATE 

0 . 92875 

GEN  SCI 

0..93232 

T/P 

45 

CATHOLIC 

0.92907 

INTESCI 

0.92713 

T/P 

51 

lEP-NO 

0.93018 

MISSING 

0.92856 

T/P 

52 

LEP-NO 

0.83100 

G/R  22 

0.89847 

T/P 

53 

CHAPl-N 

0 . 75194 

G/R  23 

0.90431 

T/P 

54 

REDUCED 

0.91650 

G/R  24 

0.95406 

T/P 

55 

FREE 

0.72745 

G/T  22 

0.72773 

T/S 

32 

INFO  NA 

0.87544 

G/T  23 

0.69148 

T/S 

33 

SCH  REF 

0.90281 

G/T  24 

0.72904 

T/S 

41 

HL-SOME 

0.85339 

G/T  25 

0.71284 

T/S 

42 

HL-ALWAY 

0.79809 

G/P  22 

0.87625 

T/S 

43 

HL-? 

0.80979 

G/P  23 

0.88131 

T/S 

51 

TVLIN-0 

0.98037 

G/P  24 

0.78673 

T/S 

52 

TV- QUAD 

0.98013 

G/P  25 

0.82195 

T/S 

53 

HW-NO 

0.95915 

G/S  22 

0.93547 

P/S 

32 

HW-YES 

0.96126 

G/S  23 

0.91639 

P/S 

33 

HWLIN-0 

0.95941 

R/T  24 

0.88414 

P/S 

41 

HWQUAD-0 

0.94546 

R/T  25 

0.86662 

P/S 

42 

1-2  HRS 

0.87252 

R/T  31 

0.88462 

P/S 

43 

3-4  HRS 

0.90544 

R/T  32 

0.89752 

P/S 

51 

5-6  HRS 

0.94207 

R/T  33 

0.86054 

P/S 

52 

7-8  HRS 

0.96277 

R/T  34 

0.87969 

P/S 

53 

9-10  HRS 

0.96101 

R/T  35 

0.89570 

G/ 

22 

> 10  HRS 

0.94668 

R/T  41 

0.87819 

G/ 

23 

B009101M 

0.75282 

R/T  42 

0.90709 

G/ 

24 

HITEM=3 

0.88494 

R/T  43 

0.92014 

G/ 

25 

HITEM=4 

0.85134 

R/T  44 

0.935,13 

G/ 

26 

MISS-2< 

0.91161 

R/T  45 

0.94157 

G/ 

27 

1-2  GRDS 

0.94676 

R/P  24 

0.88239 

R/ 

24 

3-5  GRDS 

0.92313 

R/P  25 

0.87230 

R/ 

25 

> 5 GRDS 

0.91742 

R/P  31 

0.89516 

MISSING 

0.75547 

R/P  32 

0.84568 

BOIOIOIB 

0.92025 

R/P  33 

0.88235 

ERIC 


711 


Proportion  of 
Variance 

0.87741 

0.89102 

0.86287 

0.96401 

0.94772 

0.95160 

0.92827 

0.96624 

0.96493 

0.96959 

0.95100 

0.96508 

0.95516 

0.83367 

0.83020 

0.79386 

0.83018 

0.77702 

0.76449 

0.74285 

0.82335 

0.82211 

0.82045 

0.80729 

0 .86164 

0.78413 

0.79276 

0.76487 

0.83446 

0.93520 

0.93512 

0.94594 

0.93812 

0.95391 

0.96133 

0.94122 

0 . 95970 

0.95706 

0.94346 

0.93082 

0.91747 

0.92843 

0.91823 

0.95568 

0 . 95059 

0.91993 

0.88869 

0.89713 

0.92382 

0.92393 

0.89480 

0.88571 

0.88127 


31 

32 

33 

34 

35 

36 

37 

41 

42 

43 

44 

45 

46 

47 

25 

26 

27 

31 

32 

33 

34 

35 

36 

37 

41 

42 

43 

44 

45 

46 

47 

51 

52 

53 

54 

55 

56 

57 

25 

26 

27 

31 

32 

33 

34 

35 

36 

37 

41 

42 

43 

44 

45 

46 


Table  C-14  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  8 


Proportion  of 
Variance 

0 . 89114 
0.89301 
0.87985 
0.88022 
0 . 88771 
0 . 88885 
0.89104 
0.89094 
0.88799 
0.88203 
0.95859 
0.94254 
0 . 94717 
0.94641 
0.94642 
0.95513 
0.77627 
0.72857 
0.77004 
0.77586 
0.76597 
0.77922 
0.79899 
0.76460 
0.80671 
0.81010 
0.80712 
0.82500 
0.75814 
0.73239 
0.75056 
0.72796 
0.77206 
0.76185 
0.81723 
0.81640 
0.81093 
0.82528 
0.82416 
0.77089 
0.79324 
0.82756 
0.82149 
0.77091 
0.77801 
0.76151 
0.75718 
0.77436 
0.79196 
0.73520 
0.72689 
0.75406 
0.80738 
0 . 83782 
0.85254 
0.79455 


Contrast 


T/  47 

T/  51 

T/  52 

T/.  53 

T/  54 

T/  55 

T/  56 

T/  57 

S/  23 

S/  24 

S/  25 

S/  26 

S/  27 

S/  31 

S/  32 

S/  33 

S/  34 

S/  35 

S/  36 

S/  37 

B008901N 

B008901M 

SAFE 

UNSAFE 

VRUNSAFE 

MISSING 

MOMHOM-N 

MOMHOM- ? 

DADHOM-N 

DADHOM-? 

K811001B 

K811001C 

K811001M 

K811002B 

K811002C 

K811002M 

K811003B 

K811003C 

K811003M 

K811004B 

K811004C 

K811004M 

K811005B 

K811005C 

K811005M 

K811006B 

K811006C 

K811006M 

K811007B 

K811007C 

K811007M 

K811008B 

K811008C 

K811008M 

K811101M 

K811102M 


Proportion  of 
Variance 

0.77813 
0.82041 
0.80377 
0.83346 
0.82488 
0.77455 
0.75942 
0.76203 
0.95122 
0.92274 
0.94121 
0.93674 
0.93221 
0.93462 
0.92677 
0.91612 
0.92593 
0.91833 
0.92749 
0.92679 
0.91402 
0.66302 
0.82633 
0.87104 
0.88342 
0 . 87016 
0.82638 
0.89993 
0.83372 
0 . 88324 
0.86472 
0.86809 
0.90216 
0.87834 
0.83391 
0 . 87099 
0 . 80714 
0.79994 
0 . 83722 
0.83708 
0.78722 
0.83116 
0 . 88820 
0 . 86514 
0.87145 
0.87376 
0.82705 
0.85432 
0.86851 
0.87603 
0-.81372 
0.84769 
0.81309 
0.88759 
0.72975 
0.71801 


Contrast 


K811103M 

K811104M 

K811105M 

K811106M 

K811107M 

K811108M 

K811201B 

K811201C 

K811201D 

K811201E 

K811201M 

K811401N 

K811401M 

K811501N 

K811501M 

K811601B 

K811601C 

K811601D 

K811601M 

K811602B 

K811602C 

K811602D 

K811602M 

K811603B 

K811603C 

K811603D 

K811603M 

K811604B 

K811604C 

K811604D 

K811604M 

K811605B 

K811605C 

K811605D 

K811605M 

K811606B 

K811606C 

K811606D 

K811606M 

K811609B 

K811609C 

K811609D 

K811609M 

K811610B 

K811610C 

K811610D 

K811610M 

K811611B 

K811611C 

K811611D 

K811611M 

K811612B 

K811612C 

K811612D 

K811612M 

K811701B 


Proportion  of 
Variance 

0.70933 
0.75621 
0.68818 
0.71854 
0.69382 
0.93086 
0.84019 
0.90405 
0.92124 
0.80101 
0.79853 
0.84590 
0.88161 
0.78834 
0.89711 
0.89197 
0.91423 
0.85585 
0.86567 
0.94775 
0.95656 
0.94647 
0.85469 
0.93454 
0.94630 
0.92746 
0.72829 
0.91165 
0.91959 
0.90370 
0.80173 
0.92286 
0.96265 
0.95109 
0 . 80892 
0.92998 
0.95950 
0.93549 
0.77265 
0.88827 
0.93241 
0.91500 
0.78244 
0.91748 
0 . 92316 
0.87468 
0 . 80046 
0 . 93837 
0.96312 
0 . 95086 
0.85240 
0.95105 
0 . 95529 
0.94703 
0 . 84598 
0.92557 
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Table  C-14  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  8 


Proportion  of 

Contrast  Variance 


K811701C 

0.92739 

K811701D 

0.81938 

K811701M 

0.91797 

K811702B 

0.88975 

K811702C 

0.93695 

K811702D 

0.91557 

K811702M 

0.88220 

K811703B 

0.91571 

K811703C 

0.95734 

K811703D 

0.91535 

K811703M 

0.91578 

K811704B 

0.84774 

K811704C 

0.92386 

.K811704D 

0.88198 

K811704M 

0.91560 

K811705B 

0.88071 

K811705C 

0.95244 

K811705D 

0.91098 

K811705M 

0.91713 

K811801B 

0.93819 

K811801C 

0.94240 

K811801M 

0.87096 

K811901B 

0.93512 

K811901C 

0.92890 

K811901M 

0.87312 

SMOOIOIB 

0.90525 

SMOOIOIC 

0.91770 

SMOOIOID 

0.93306 

SMOOIOIM 

0.95285 

SM00201B 

0.93476 

SM00201C 

0.93711 

SM00201D 

0.93611 

SM00201M 

0.95767 

SM00301B 

0.90366 

SM00301C 

0.90169 

SM00301D 

0.90118 

SM00301M 

0.94894 

SM00401B 

0 . 88398 

SM00401C 

0.90140 

SM00401D 

0.87922 

SM00401M 

0 . 95275 

SM00501B 

0 . 85855 

SM00501C 

0.87566 

SM00501D 

0 . 86682 

SM00501M 

0 . 95177 

B009701B 

0.90354 

B009701C 

0 . 89062 

B009701D 

0 . 93025 

B009701E 

0.94423. 

B009701F 

0.88957 

B009701M 

0.80372 

B009801B 

0.94022 

B009801C 

0 . 96009 

B009801D 

0.95302 

B009801E 

0.93544 

B009801M 

0.75458 

Proportion  of 

Contrast  Variance 


B009601B 

0.87602 

B009601N 

0.86451 

B009601D 

0.83509 

B009601E 

0.87279 

B009602B 

0.93300 

B009602N 

0.93749 

B009602D 

0.84613 

B009602M 

0.85463 

K812101N 

,0.88273 

K812101M 

0.68304 

K812201B 

0.90121 

K812201C 

0.97806 

K812201D 

0.97145 

K812201E 

0.96189 

K812201F 

0.95603 

K812201G 

0.94666 

K812201M 

0.86237 

K811613B 

0.84497 

K811613C 

0.88635 

K811613D 

0.81316 

K811613M 

0.80864 

K811614B 

0.84704 

K811614C 

0.90033 

K811614D 

0.82387 

K811614M 

0 . 84082 

K811615B 

0.95950 

K811615C 

0.96202 

K811615D 

0.95256 

K811615M 

0.89083 

C031603N 

0.90975 

C031603M 

0.98092 

C031607N 

0.92014 

C031607M 

0.97281 

C031610N 

0.93440 

C031610M 

0 . 96670 

C031606N 

0 . 91719 

C031606M 

0 . 95685 

C035701N 

0 . 91734 

C035701M 

0 . 95147 

C035702N 

0 . 90181 

C035702M 

0.96955 

C035703N 

0 . 90879 

C035703M 

0 . 95440 

C037201M 

0.92725 

C037202M 

0 . 94087 

C037204M 

0 . 91201 

C037205M 

0 . 91814 

C037206M 

0.94486 

C037301M 

0 . 96584 

C037302M 

0.95274 

C037304M 

0.92484 

C037305M 

0.93716 

C036601B 

0 . 88832 

C036601C 

0.93494 

C036601D 

0.95643 

C036601M 

0.98130 

Contrast 

C032207B 

C032207N 

C032207M 

C032209B 

C032209N 

C032209M 

C032210B 

C032210N 

C032210M 

C032211B 

C032211N 

C032211M 

C037701B 

C037701C 

C037701D 

C037701M 

C037702B 

C037702C 

C037702D 

C037702M 

C037703B 

C037703C 

C037703D 

C037704B 

C037704C 

C037704D 

C037705B 

C037705C 

C037705D 

C032402B 

C032402C 

C032402D 

C032401B 

C032401C 

C032401D 

C032404B 

C032404C 

C032404D 

C032406B 

C032406C 

C032406D 

C032407B 

C032407C 

C032408B 

C032408C 

C032408D 

C032409B 

C032409C 

C032409D 

C032410B 

C032410C 

C032410D 

C032411B 

C032411C 

C032411D 

C032412B 


Proportion  of 
Variance 

0.92927 

0.94546 

0.97639 

0.92106 

0.91132 

0.98314 

0.92488 

0.92843 

0.97860 

0.91891 

0.92395 

0.98808 

0.90725 

0.93778 

0.89634 

0.97900 

0.94639 

0.93730 

0.94056 

0.98515 

0.92938 

0.93699 

0.94122 

0.91366 

0.92668 

0.94508 

0.91635 

0.92357 

0.90238 

0.93579 

0.94391 

0.94077 

0.92283 

0.94059 

0.93269 

0.92135 

0.94445 

0.92768 

0.90498 

0.94860 

0.95961 

0.91251 

0.92244 

0.90869 

0.94606 

0.95375 

0.93567 

0.95068 

0.91318 

0.95175 

0.96093 

0.96021 

0.95345 

0.95621 

0.95241 

0.93736 


Table  C-14  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  8 


Contrast 

Proportion  of 
Variance 

C032412C 

0.94614 

C032413B 

0.91707 

C032413C 

0.94331 

C032413D 

0.95711 

C032414B 

0.94610 

C032414C 

0.95588 

C032414D 

0.91607 

C032415B 

0.94094 

C032415C 

0.92816 

C032502B 

0.91340 

C032502C 

0.93605 

C032503B 

0.93127 

C032503C 

0.93640 

C032505B 

0.92988 

C032505C 

0.93405 

C032506B 

0.93706 

C032506C 

0.90986 

C032506D 

0.93664 

C033601B 

0.93383 

C033601C 

0.94904 

C033601D 

0.91813 

C036501B 

0.93041 

C036501C 

0.89186 

C037801B 

0 . 92049 

C037801C 

0 .94056 

C037801D 

0.92005 

C037801E 

0.89936 

C037801F 

0.90621 

C037801G 

0.8998,2 

C037801H 

0 . 97713 

C038001B 

0.92061 

C038001C 

0.93432 

C038001D 

0.91998 

C038001E 

0.92972 

C038001F 

0.95733 

C038301N 

0.89979 

C038301M 

0.97806 

C038801N 

0 . 90879 

C038801M 

0.96647 

C034101M 

0 . 92820 

C034102M 

0.87312 

C034103M 

0.91047 

C034104M 

0.90529 

C034105M 

0 . 92680 

C034106M 

0.91053 

C034107M 

0.90289 

C034108M 

0.92767 

C034109M 

0.90994 

T055901B 

0.88802 

T055901M 

0.98465 

T056001B 

0.89223 

T056001C 

0.86428 

T056001D 

0.88436 

T056001E 

0 . 87119 

T056001M 

0.96663 

T040301B 

0.88967 

O 


Contrast 

Proportion  of 
Variance 

T040301C 

0.91439 

T040301D 

0.92707 

T040301E 

0.93078 

T040301M 

0.98985 

T056201B 

0.92043 

T056201C 

0.90183 

T056201D 

0.88880 

T056201E 

0.88756 

T056201F 

0.90292 

T056201M 

0.96554 

T040501N 

0.89775 

T040501C 

0.89466 

T040501M 

0.92269 

T040507N 

0.93038 

T040507C 

0.90690 

T040507M 

0.94888 

T040508N 

0 . 89248 

T040508C 

0.89449 

T040508M 

0.91750 

T040505N 

0.89959 

T040505C 

0.91096 

T040505M 

0.90737 

T056301C 

0.95510 

T056301D 

0.94411 

T056301E 

0.93838 

T056301F 

0.88421 

T056301G 

0.93448 

T040701M 

0.89623 

T040706M 

0.90710 

T040707M 

0.89193 

T040710M 

0.89837 

T040711M 

0.89568 

T040712M 

0.89824 

T040713M 

0.91887 

T040708M 

0.87922 

T040709M 

0.89708 

T040705M 

0.88932 

T040801M 

0.87570 

T040807M 

0 . 89148 

T040808M 

0 .88910 

T040814M 

0.88568 

T040815M 

0.90948 

T040816M 

0.89814 

T040817M 

0 . 91779 

T040809M 

0 . 87783 

T040810M 

0 . 91710 

T040811M 

0.89671 

T040812M 

0.88915 

T040813M 

0 . 91308 

T040805M 

0.88688 

T040806M 

0.90519 

T056401M 

0.90580 

T056402M 

0.90364 

T056403M 

0.91130 

T056405M 

0.89011 

T056413M 

0.90415 

714 


Proportion  of 


Contrast  Variance 


T056414M 

0.89906 

T056415M 

0.88978 

T056406M 

0.90689 

T056407M 

0.83465 

T056408M 

0 . 89545 

T056409M 

0.89098 

T056410M 

0.89696 

T056411M 

0.91324 

T056412M 

0.91635 

T056701M 

0.90719 

T056702M 

0.90765 

T056703M 

0.88362 

T056704M 

0.89440 

T056705M 

0.88548 

T056706M 

0.89563 

T056707M 

0.89044 

T056708M 

0.89673 

T056709M 

0.89234 

T056710M 

0 . 89310 

T056711M 

0.89591 

T056712M 

0.88572 

T056713M 

0.90095 

T041201B 

0.92948 

T041201C 

0.92377 

T041201D 

0.90187 

T041201M 

0.97482 

T056801B 

0.90269 

T056801C 

0.89685 

T056801D 

0.93572 

T056801E 

0.92502 

T056801F 

0.92079 

T056801M 

0.97882 

T060301M 

0.89871 

T060311M 

0.88713 

T060321M 

0.91015 

T060331M 

0.90785 

T060341M 

0.89674 

T060302M 

0.91440 

T060312M 

0.88867 

T060322M 

0.90793 

T060332M 

0.91563 

T060342M 

0.89156 

T060303M 

0 . 90771 

T060313M 

0 . 91891 

T060323M 

0.91814 

T060333M 

0.93180 

T060343M 

0 . 88995 

T060304M 

0.90672 

T060314M 

0.88527 

T060324M 

0.92512 

T060334M 

0.89411 

T060344M 

0.87520 

T060305M 

0 . 89475 

T060315M 

0.91378 

T060325M 

0.90676 

T060335M 

0.89944 

663 


Table  C-14  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  8 


Proportion  of 

Contrast Variance 


T060345M 

0.90127 

T060306M 

0.91021 

T060316M 

0.89986 

T060326M 

0.89358 

T060336M 

0.90101 

T060346M 

0.88824 

T060307M 

0.93028 

T060317M 

0 . 95089 

T060327M 

0 . 95911 

T060337M 

0.93385 

T060347M 

0.94106 

T060401M 

0.89735 

T060402M 

0.90556 

T060403M 

0 . 90690 

T060404M 

0.90239 

T060405M 

0.88399 

T060501N 

0 . 88255 

T060501M 

0.96558 

T060601B 

0.89817 

T060601C 

0.90725 

T060601D 

0.91362 

T060601M 

0.98229 

T060602B 

0.91039 

T060602C 

0.94621 

T060602D 

0.93498 

T060602M 

0.97138 

T060603B 

0.92096 

T060603C 

0.92704 

T060603D 

0.90280 

T060603M 

0.99358 

T060604B 

0.92797 

T060604C 

0.91529 

T060604D 

0.88505 

T060604M 

0.98204 

T060605B 

0.87990 

T060605C 

0 . 94006 

T060605D 

0.94583 

T060606B 

0.91430 

T060606C 

0.93240 

T060606D 

0.93459 

T060606M 

0 . 96897 

T060607B  ■ 

0.93251 

T060607C 

0.91844 

T060607D 

0 . 89907 

T060607M 

0.98348 

T060608B 

0.92186 

T060608C 

0.92566 

T060608D 

0.89098 

T060608M 

0.98679 

T060609B 

0 . 94387 

T060609C 

0.94775 

T060609D 

0.90733 

T060609M 

0.98983 

T060610B 

0.91969 

T060610C 

0 . 93803 

T060610D 

0.92345 

Contrast 

Proportion  of 
Variance 

T060611B 

0.92131 

T060611C 

0.94551 

T060611D 

0.95990 

T060611M 

0.97921 

T060701B 

0.87986 

T060701C 

0.91720 

T060701D 

0.89237 

T060701M 

0.98628 

T060702B 

0.92521 

T060702C 

0.92786 

T060702D 

0.88744 

T060703B 

0.93667 

T060703C 

0.96062 

T060703D 

0.92577 

T060704B 

0.92499 

T060704C 

0.94650 

T060704D 

0.96097 

T060704M 

0.98845 

T060705B 

0.91501 

T060705C 

0.91963 

T060705D 

0.94025 

T060801B 

0.92565 

T060801C 

0.94636 

T060801M  ' 

0.98460 

T060901B 

0.92840 

T060901C 

0.94011 

T061001N 

0.92429 

T061101B 

0.90209 

T061101C 

0.89796 

T061102B 

0.90173 

T061102C 

0.88588 

T061103B 

0.89462 

T061103C 

0.88815 

T061104B 

0.89675 

T061104C 

0.90901 

T061105B 

0.90915 

T061105C 

0.89809 

T061106B 

0.89719 

T061106C 

0.90879 

T061106M 

0.99140 

T061107B 

0.89008 

T061107C 

0.88150 

T061108B 

0.91325 

T061108C 

0.91792 

T061109B 

0.91804 

T061109C 

0.93857 

T061109M 

0.97808 

T061201N 

0.89944 

T061201M 

0.97288 

T061301B 

0.92181 

T061301C 

0.92146 

T061301D 

0.88518 

T061301E 

0.87885 

T061302B 

0.89660 

T061302C 

0.89068 

T061302D 

0.89932 

Proportion  of 

Contrast  VsiHfinrp 

T061302E 

0.90358 

T061302M 

0.98735 

T061303B 

0.92772 

T061303C 

0.91672 

T061303D 

0.92102 

T061303E 

0.89028 

T061303M 

0.97771 

T061304B 

0.90561 

T061304C 

0.92062 

T061304D 

0 . 91873 

T061304E 

0.90233 

T061305B 

0.91260 

T061305C 

0.90751 

T061305D 

0.91166 

T061305E 

0.94292 

T061306B 

0.90959 

T061306C 

0.90314 

T061306D 

0.91100 

T061306E 

0 . 92821 

T061307B 

0.91269 

T061307C 

0.89753 

T061307D 

0.92324 

T061307E 

0.92166 

T061308B 

0 . 90178 

T061308C 

0 . 89947 

T061308D 

0.87906 

T061308E 

0.92257 

T061309B 

0.88125 

T061309C 

0.90314 

T061309D 

0.88450 

T061309E 

0.88346 

T061310B 

0.91339 

T061310C 

0.89561 

T061310D 

0.88164 

T061310E 

0.87982 

T061401B. 

0.93338 

T061401C 

0.93784 

T061401D 

0.87774 

T061501B 

0.89898 

T061501C 

0.87609 

T061501D 

0.93697 

T061501E 

0.91172 

T061501F 

0.90437 

T061501M 

0.92621 

T061601M 

0.88164 

T061611M 

0.90275 

T061621M 

0.90213 

T061631M 

0.88394 

T061641M 

0.90154 

T061651M 

0.91183 

T061701B 

0.87908 

T061701C 

0.87702 

T061701D 

.0.87003 

T061701E 

0.86902 

T061701M 

0.93701 

T061801B 

0.89685 

Table  C-14  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  8 


Contrast 


•T061801C 

T061801D 

T061801E 

T061801M 

T0.61901B 

T061901C 

•T061901M 

T062001B 

T062001C 

T062001D 

T062001M 

T062002B 

T062002C 

T062002D 

T062002M 

T062003B 

T062003C 

T062003D 

T062003M 

T062101B 

T062101C 

T062101D 

T062101M 

T062201M 

T0.62202M 

T062203M 

T062204M 

T062205M 

T062206M 

T062207M 

T062208M 

T062209M 

T062210M 

T062301B 

T062301C 

T062301D 

T062301E 

T062301M 

T062401B 

T062401C 


Proportion  of 
Variance 

0.82298 

0.82136 

0.87751 

0.88743 

0.97672 

0.89348 

0.95575 

0.92394 

0.91700 

0.90789 

0.92574 

0.91343 

0.88801 

0.89329 

0.92914 

0.91379 

0.91518 

0.89893 

0.92391 

0.88537 

0 .91298 

0.91918 

0 . 94411 

0.88523 

0.90225 

0.87435 

0 . 87554 

0.91436 

0.90197 

0.90015 

0.88440 

0.87189 

0.88852 

0.87946 

0.91774 

0.92148 

0.89088 

0.96040 

0.84979 

0.87496 


Contrast 

T062401D 

T062401E 

T062401M 

C034201B 

C034201C 

C034402N 

C034401N 

C034510D 

C034514B 

C039401M 

C039501M 

T062501E 

T062601B 

T062601C 

T062801N 

NATLUNCL 

REMREADL 

REMMATHL 

NATLUN8L 

REMREA8L 

REMMAT8L 


Proportion  of 
Variance 

0.83843 
0 . 87456 
0.89555 
0.93437 
0.95508 
0.91768 
0.92105 
0 . 88851 
0.91214 
0 . 90840 
0.88955 
0.93702 
0.88934 
0.90431 
0.90888 
0.93080 
0.95036 
0.93477 
0.95035 
0.94179 
0.91575 


Contrast 


Proportion  of 
Variance 


665 


Contrast 


FEMALE 
BLACK 
HISPANIC 
ASIAN 
MEXICAN 
PUER  RIC 
CUBN^OTH 
HISP-? 
NON  MSA 
MID  CTY5 
FR/BTWN5 
SML  TWN5 
RURAL 5 
URBAN  FR 
MED  CITY 
SM  PLACE 
HS  GRAD 
POST  HS 
COL  GRAD 
PARED-? 

S EAST 

CENTRAL 

WEST 

PRIVATE 

CATHOLIC 

lEP-NO 

LEP-NO 

CHAPl-N 

REDUCED 

FREE 

INFO  NA 

SCH  REF 

HL-SOME 

HL-ALWAY 

HL-? 

TVLIN-0 

TV- QUAD 

HW-NO 

HW-YES 

HWLIN-0 

HWQUAD-0 

HITEM=3 

HITEM=4 

MISS-2< 

USA  >5 

USA  3-5 

USA  <3 

USA-? 

SNRM-LIN 

B007302B 

B007302C 

B007302D 

B007302E 

B007302F 

B007302G 

B007302H 


Table  C-15 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  12 


Proportion  of 
Variance 

0.92370 

0.91671 

0.96148 

0.93361 

0.92939 

0.97662 

0.94184 

0.92808 

0.91456 

0.93434 

0.94074 

0.95555 

0.94797 

0.90644 

0.85799 

0.91587 

0.93832 

0.94431 

0.94447 

0.95349 

0.87537 

0 . 87869 

0.89809 

0.90600 

0.91113 

0.96858 

0.84055 

0.80994 

0.96889 

0.81843 

0.84872 

0.88282 

0.87173 

0.79104 

0.74483 

0.97823 

0.97682 

0 . 97714 

0.97841 

0.95670 

0.93911 

0.89323 

0.85631 

0.95667 

0.85566 

0.85481 

0.87402 

0.85582 

0.85074 

0.93970 

0.94259 

0.92964 

0.94782 

0.96231 

0.95909 

0.93841 


Proportion  of 

Contrast  Variance 


B007302M 

0.89368 

STGRDl-2 

0.93694 

STGRD3-5 

0.93985 

STGRD6> 

0.93795 

DIS0HOM2 

0.89580 

DIS0HOM3 

0.92398 

DIS0HOM4 

0.85025 

1-2  WEEK 

0.95179 

1-2  MTH 

0.95974 

NEVER 

0.96102 

NO  COMP 

0.93694 

MISSING 

0.86829 

PGS>5  ‘ 

0.83576 

PGS>10 

0.83543 

ACADEMIC 

0.74011 

VOC/TECH 

0.88858 

OTHERPGM 

0.93125 

HS  PGM-? 

0 . 87703 

SEMENG-'" 

0.85785 

#ENG-LIN 

0.86187 

SEMMAT-'" 

0.89980 

#MAT-LIN 

0.85762 

SEMSCI-'" 

0.94353 

SEMSCI-^ 

0.94391 

SEMSCI-'" 

0.92937 

SEMSCI-^ 

0.94424 

#SCI-LIN 

0.94928 

SEMHIS-'" 

0.87145 

#HIS-LIN 

0.81282 

SEMFLG-^ 

0.87461 

#FLG-LIN 

0.73498 

SEMVOC-'" 

0.83514 

#VOC-LIN 

0.83192 

SEMART-'" 

0.85524 

#ART-LIN 

0.84026 

G/R  22 

0.90404 

G/R  23 

0.90937 

G/R  24 

0.94198 

G/T  22 

0.72103 

G/T  23 

0.73591 

G/T  24 

0.73911 

G/T  25 

0.75204 

G/P  22 

0.88244 

G/P  23 

0.93367 

G/P  24 

0.84716 

G/P  25 

0.87283 

G/S  22 

0.93957 

G/S  23 

0.93157 

R/T  24 

0.86645 

R/T  25 

0.88815 

R/T  31 

0.88006 

R/T  32 

0.90237 

R/T  33 

0.87748 

R/T  34 

0.88682 

R/T  35 

0.89641 

R/T  41 

0.91799 

Proportion  of 

Contrast  Variance 


R/T 

42 

0 . 90707 

R/T 

43 

0.90569 

R/T 

44 

0.89637 

R/T 

45 

0.94571 

R/P 

24 

0.89406 

R/P 

25 

0.90413 

R/P 

31 

0.89725 

R/P 

32 

0.84060 

R/P 

33 

0.89209 

R/P 

34 

0.90020 

R/P 

35 

0.90331 

R/P 

41 

0.85040 

R/P 

42 

0.93317 

R/P 

43 

0.94337 

R/P 

44 

0.93201 

R/P 

45 

0.85683 

R/S 

31 

0.96840 

R/S 

32 

.0.94832 

R/S 

33 

0.96108 

R/S 

41 

0.91717 

R/S 

42 

0.97077 

R/S 

43 

0.96034 

T/P 

25 

0.78555 

T/P 

31 

0.77884 

T/P 

32 

0.80087 

T/P 

33 

0.85271 

T/P 

34 

0.80212 

T/P 

35 

0.77866 

T/P 

41 

0.79150 

T/P 

42 

0.87999 

T/P 

43 

0.76779 

T/P 

44 

0.78170 

T/P 

45 

0.81952 

T/P 

51 

0.83260 

T/P 

52 

0.78216 

T/P 

53 

0.77941 

T/P 

54 

0.82295 

T/P 

55 

0.86631 

T/S 

32 

0.93111 

T/S 

33 

0.92985 

T/S 

41 

0.93703 

T/S 

42 

0.91521 

T/S 

43 

0.95788 

T/S 

51 

0.95006 

P/S 

32 

0.95860 

P/S 

33 

0.93211 

P/S 

41 

0.94838 

P/S 

42 

0.92213 

P/S 

43 

0.93854 

P/S 

51 

0.91858 

P/S 

52 

0.96626 

P/S 

53 

0.94966 

G/S 

22 

0.87359 

Contrast 

G/S  23 
G/S  24 
R/S  24 
R/S  25 
R/S  31 
R/S  32 
R/S  33 
R/S  34 
R/S  35 
R/S  41 
R/S  42 
R/S  43 
R/S  44 
R/S  45 
P/S  25 
P/S  31 
P/S  32 
P/S  33 
P/S  34 
P/S  35 
P/S  41 
P/S  42 
P/S  43 
P/S  44 
P/S  45 
P/S  51 
P/S  52 
P/S  53 
P/S  54 
P/S  55 
T/S  25 
T/S  31 
T/S  32 
T/S  33 
T/S  34 
T/S  35 
T/S  41 
T/S  42 
T/S  43 
T/S  44 
T/S  45 
T/S  51 
T/S  52 
T/S  53 
T/S  54 
T/S  55 
S/S  23 
S/S  24 
S/S  25 
S/S  31 
S/S  32 
S/S  33 
S/S  34 
S/S  35 
B008901N 
B008901M 


o 


Table  C-15  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  12 


Proportion  of 
Variance 

0.85199 
0.80521 
0.84972 
0.89377 
0.88767 
0.91324 
0.83528 
0.88995 
0.90079 
0.91950 
0.92783 
0.95621 
0.94445 
0.94598 
0.84160 
0.81919 
0.82187 
0.86127 
0.80452 
0.85549 
0.86807 
0.87586 
0.84197 
0 . 82647 
0.82360 
0.82931 
0.88706 
0 . 92145 
0.92535 
0.91986 
0.82508 
0.82858 
0.80646 
0.85515 
0.82833 
0.83691 
0.76720 
0.79785 
0.78394 
0.81266 
0 . 81070 
0.81065 
0.84731 
0.84975 
0.85432 
0.84521 
0.94833 
0.93453 
0.91953 
0.90401 
0.92777 
0.90733 
0.90341 
0.89551 
0.96035 
0.73193 


Contrast 

Proportion  of 
Variance 

SAFE 

0.83758 

UNSAFE 

0.91634 

VRUNSAFE 

0.89039 

MISSING 

0.86977 

MOMHOM-N 

0.79818 

MOMHOM- ? 

0 . 84249 

■ DADHOM-N 

0.80065 

DADHOM- ? 

0.83309 

K811001B 

0.87640 

K811001C 

0.86949 

K811001M 

0.96926 

K811002B 

0.89254 

K811002C 

0 . 85703 

K811002M 

0 . 95790 

K811003B 

0.81128 

K811003C 

0.80332 

K811003M 

0.91841 

K811004B 

0 . 83191 

K811004C 

0.79424 

K811004M 

0 . 87589 

K811005B 

0.89915 

K811005C 

0.84462 

K811005M 

0.93623 

K811006B 

0.83533 

K811006C 

0.80193 

K811006M 

0.93241 

K811007B 

0.88119 

K811007C 

0.88794 

K811007M 

0.92876 

K811008B 

0.84982 

K811008C 

0.81528 

K811008M 

0.93534 

K811501N 

0.83972 

K811501M 

0.88107 

K811601B 

0.91457 

K811601C 

0.93836 

K811601D 

0.86253 

K811601M 

0.90577 

K811602B 

0.96573 

K811602C 

0.97587 

K811602D 

0.96610 

K811602M 

0.90736 

K811603B 

0.95509 

K811603C 

0.96497 

K811603D 

0.94843 

K811603M 

0.86568 

K811604B 

0.86856 

K811604C 

0.92230 

K811604D 

0.82110 

K811604M 

0.80114 

K811605B 

0.96389 

K811605C 

0.97.974 

K811605D 

0.97665 

K811605M 

0.90457 

K811606B 

0.95720 

K811606C 

0.97895 

Contrast 

Proportion  of 
Variance 

K811606D 

0.96824 

K811606M 

0.88656 

K811609B 

0.88984 

K811609C 

0.94882 

K811609D 

0.90147 

K811609M 

0.85823 

K811610B 

0 . 90043 

K811610C 

0.92427 

K811610D 

0.79174 

K811610M 

0 . 88371 

K811611B 

0.97180 

K811611C 

0.97730 

K811611D 

0 . 97093 

K811611M 

0.89673 

K811612B 

0.94892 

K811612C 

0.97547 

K811612D 

0.96428 

K811612M 

0 .91000 

K811701B 

0.96680 

K811701C 

0.96952 

K811701D 

0.81164 

K811701M 

0.93645 

K811702B 

0.87026 

K811702C 

0.90906 

K811702D 

0.87970 

K811702M 

0.92286 

K811703B 

0.92065 

K811703C 

0.96463 

K811703D 

0.89293 

K811703M 

0.93180 

K811704B 

0.86213 

K811704C 

0.94034 

K811704D 

0.88453 

K811704M 

0.92997 

K811705B- 

0.92549 

K811705C 

0.96879 

K811705D 

0.94646 

K811705M 

0.94342 

K811801B 

0.95538 

K811801C 

0.95294 

K811801M 

0.91146 

K811901B 

0.95389 

K811901C 

0.94725 

K811901M 

0.90236 

SMOOIOIB 

0.93541 

SMOOIOIC 

0.94607 

SMOOIOID 

0.94310 

SMOOIOIM 

0.93669 

SM00201B 

0.94988 

SM00201C 

0 . 94372 

SM00201D 

0.93606 

. SM00201M 

0.92801 

SM00301B 

0.96646 

SM00301C 

0.95724 

SM00301D 

0.94827 

SM00301M 

0.95431 
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Table  C-15  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  12 


Contrast 

Proportion  of 
Variance 

SM00401B 

0.93849 

SM00401C 

0.95563 

SM00401D 

0.94106 

SM00401M 

0.95862 

SM00501B 

0.89908 

SM00501C 

0.91708 

SM00501D 

0.86213 

SM00501M 

0.94235 

B009601B 

0.96681 

B009601N 

0.82588 

B009601D 

0.82668 

B009601E 

0.91039 

B009602B 

0.97020 

B009602N 

0 . 91670 

B009602D 

0.81088 

B009602M 

0.89009 

K812201B 

0 . 95074 

K812201C 

0.95852 

K812201D 

0 . 96820 

K812201E 

0.95870 

K812201F 

0 . 96426 

K812201G 

0 . 92004 

K812201M 

0.76695 

K811613B 

0.78967 

K811613C 

0.81480 

K811613D 

0 . 81977 

K811613M 

0 . 90693 

K811614B 

0.80434 

K811614C 

0.83632 

K811614D 

0.81649 

K811614M 

0.92128 

K811615B 

0.95715 

K811615C 

0.96735 

K811615D 

0.95802 

K811615M 

0.90318 

B008301D 

0.91047 

B008301M 

0.85479 

B009901B 

0.90169 

B009901C 

0.88406 

B009901D 

0.95840 

B009901E 

0.96738 

B009901F 

0.93242 

B009901M 

0.82910 

B005501B 

0.95315 

B005501C 

0.95268 

B005501D 

0.90958 

B005501E 

0.96631 

B005501F 

0.96104 

B005501M 

0.89563 

OFFICE  W 

0.93957 

TECHNIC 

0.96790 

PROT  SRV 

0 . 97745 

SALES 

0.97623 

OWNER 

0.96605 

SERV  WRK 

0.96570 

SKILL  T 

0 . 96972 

Contrast 

Proportion 

Variance 

LABORER 

0.97106 

OPERATOR 

0.97540 

FARMER 

0.97340 

HOMEMAKE 

0.84191 

MANAGER 

0 . 97479 

MILITARY 

0.95518 

TEACHER 

0.93503 

PROFESSl 

0.94833 

PROFESS2 

0.96191 

BOllAOlN 

0 . 83488 

OFFICE 

0.97489 

TECHNIC 

0.96958 

PROT  SRV 

0 . 97817 

SALES 

0.97251 

OWNER 

0 . 93699 

SERV  WRK 

0 . 97771 

SKILL  TR 

0.95835 

LABORER 

0.96193 

OPERATOR 

0.96862 

FARMER 

0 . 96916 

HOMEMAKE 

0.92559 

MANAGER 

0 . 95432 

MILITARY 

0 . 97175 

TEACHER 

0 . 96287 

PROFESSl 

0 . 95270 

PROFESS2 

0.95053 

B012A02N 

0 .80717 

K812301M 

0.68403 

K812302M 

0.72750 

K812303M 

0.70247 

K812304M 

0.77563 

K812305M 

0.70288 

K812306M 

0.67864 

K812307M 

0.70809 

K812308M 

0.73819 

K812309M 

0.90994 

K812401N 

0 . 87571 

K812401M 

0.73201 

K812501B 

0.93090 

K812501C 

0.94349 

K812501D 

0.87892 

K812501M 

0.79450 

K812502B 

0.95817 

K812502C 

0.93269 

K812502D 

0.92930 

K812502M 

0.77542 

K812503B 

0.83251 

K812503C 

0.90384 

K812503D 

0 . 89370 

K812503M 

0.73458 

K812504B 

0.96720 

K812504C 

0.93237 

K812504D 

0.90946 

K812504M 

0.75624 

K812505B 

0.93971 

K812505C 

0.96015 

?19 


Proportion  of 

Contrast 

Variance 

K812505D 

0.91924 

K812505M 

0.81853 

K812506B 

0.93685 

K812506C 

0.96164 

K812506D 

0.91692 

K812506M 

0.81541 

K812507B 

0.95634 

K812507C 

0.93825 

K812507D 

0.92636 

K812507M 

0.75786 

K812508B 

0.95876 

K812508C 

0.94943 

K812508D 

0.92637 

K812508M 

0.92681 

K812509B 

0.96423 

K812509C 

0.94068 

K812509D 

0.93203 

K812509M 

0.92891 

K812510B 

0.95701 

K812510C 

0 . 97071 

K812510D 

0.90151 

K812510M 

0.81960 

K812601N 

0.76508 

K812601M 

0.85884 

K812701N 

0 .94118 

K812701M 

0.72392 

K811616B 

0.88195 

K811616C 

0 . 95726 

K811616D 

0.86971 

K811616M 

0.90145 

C035701N 

0.87089 

C035701M 

0.93561 

C035702N 

0.88353 

C035702M 

0.96946 

C035703N 

0.88092 

C035703M 

0.90409 

C037201M 

0 . 92494 

C037202M 

0.91093 

C037204M 

0 . 88947 

C037205M 

0.88924 

C037206M 

0 . 88132 

C037301M 

0.97333 

C037302M 

0 . 96547 

C037304M 

0.86899 

C037305M 

0.91604 

C036601B 

0 . 89973 

C036601C 

0 . 90271 

C036601D 

0 . 94276 

C036601M 

0 . 97157 

C032207B 

0.92786 

C032207N 

0 . 92407 

C032207M 

0.92511 

C032209B 

0 . 91965 

C032209N 

0.92544 

C032209M 

0 . 95453 

C032210B 

0.94360 

Table  C-15  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  12 


Contrast 


Proportion  of 
Variance 


Contrast 


Proportion  of 
Variance 


Contrast 


Proportion  of 
Variance 


C032210N 

0.95198 

C032210M 

0 . 93908 

C032211B 

0.88500 

C032211N 

0.89029 

C032211M 

0.95735 

C037701B 

0.86986 

C037701C 

0 . 90194 

C037701D 

0.88184 

C037701M 

0.96339 

C037702B 

0.89756 

C037702C 

0.91838 

C037702D 

0.90530 

C037702M 

0.98220 

C037703B 

0.89745 

C037703C 

0.92208 

C037703D 

0.90642 

C037704B 

0.88329 

C037704C 

0.87431 

C037704D 

0.90198 

C037704M 

0.96324 

C037705B 

0.89701 

C037705C 

0.89818 

C037705D 

0.90113 

C032402B 

0.93144 

C032402C 

0.93392 

C032402D 

0.91518 

C032402M 

0.99407 

C032401B 

0.91072 

C032401C 

0.94322 

C032401D 

0.89027 

C032404B 

0.89397 

C032404C 

0.94549 

C032404D 

0.91661 

C032406B 

0.89388 

C032406C 

0.95032 

C032406D 

0.94029 

C032407B 

0.88643 

C032407C 

0.86886 

C032408B 

0.89443 

C032408C 

0.96359 

C032408D 

0.95832 

C032409B 

0.92367 

C032409C 

0.93134 

C032409D 

0.91390 

C032410B 

0.91959 

C032410C 

0.93374 

C032410D 

0.90272 

C032410M 

0.98139 

C032411B 

0.93842 

C032411C 

0.94358 

C032411D 

0 . 88421 

C032412B 

0.93187 

C032412C 

0.93695 

C032412D 

0.91771 

C032413B 

0.90251 

C032413C 

0.94961 

C032413D 

0.95037 

C032414B  . 

0.91331  , 

C032414C 

0.94297 

C032414D 

0.88909 

C032415B 

0.92753 

C032415C 

0.95380 

C032415D 

0.92709 

C032502B 

0.90890 

C032502C 

0.90013 

C032502D 

0.90145 

C032502M 

0.96764 

C032503B 

0.89695 

C032503C 

0.92592 

C032505B 

0.90278 

C032505C 

0.90078 

C032505D 

0.91001 

G032506B 

0.89881 

C032506C 

0.87640 

C032506D 

0.86800 

C033601B 

0.92369 

C033601C 

0.93328 

C033601D 

0.91139 

C033601M 

0.97710 

C036501B 

0.90250 

C036501C 

0 . 89498 

C036501M 

0.97172 

C037801B 

0.89545 

C037801C 

0.90352 

C037801D 

0.90447 

C037801E 

0 . 91240 

C037801F 

0.90460 

C037801G 

0.89732 

C038001B 

0.85284 

C038001C 

0 . 87525 

C038001D 

0.91860 

C038001M 

0.98010 

C038301N 

0 . 87718 

C038301M 

0.91358 

C038801N 

0.90183 

C038801M 

0.89507 

C034101M 

0.88584 

C034102M 

0 . 90191 

C034103M 

0.87030 

C034104M 

0.88278 

C034105M 

0.87628 

C034106M 

0.87212 

C034107M 

0.88182 

C034108M 

0.90287 

C034109M 

0.89353 

C037203M 

0.90966 

C037306M 

0.96368 

C035002N 

0.93660 

C035002M 

0.97875 

C035003N 

0.92974 

C035003M 

0.97412 

C035006N 

0.87744 

C035006M 

0 . 98034 

C035007N 

0.88632 

C035007M 

0.95328 

C040201C 

0.88300 

C040-201D 

0.94551 

C040201E 

0.94396 

C040201F 

0.89375 

C040202C 

0.89626 

C040202D 

0.95294 

C040202E 

0.93950 

C040202F 

0.87251 

C040203E 

0.93462 

C040203F 

0.94699 

C040204B 

0.87484 

C040204C 

0.88743 

C040204D 

0.88296 

C040204F 

0.91571 

C040204M 

0.95036 

C040301M 

0.92424 

C040302M 

,0.88503 

C040303M 

0.88238 

C040304M 

0.91496 

C040305M 

0.86109 

C040306M 

0.89956 

C040307M 

0.84836 

C040308M 

0.88209 

C040309M 

0.86589 

C040310M 

0 . 88911 

C040311M 

0.90474 

C040401N 

0.94208 

C040402N 

0.92615 

C040402M 

0.94019 

C040403N 

0.94880 

C040404N 

0.93593 

C040501M 

0.88498 

C040502M 

0.87825 

C040503M 

0 . 88078 

C040504M 

0.90028 

C040505M 

0.84857 

C040506M 

0 . 89174 

C040601M 

0.87994 

C040602M 

0 . 86801 

C040603M 

0.88405 

C040604M 

0.88632 

C040605M 

0.85646 

C040701M 

0.93398 

C040702M 

0.90968 

C040703M 

0.89486 

C040704M 

0.88840 

C040705M 

0.91671 

C040801B 

0.89025 

C040801C 

0.88122 

Table  C-15  (continued) 

Proportion  of  Variance  of  the  Conditioning  Variable  Contrasts  Accounted  for 
by  the  Principal  Components  Used  in  the  Conditioning  Model  for 
Science  Main  Conditioning  Variables,  Grade  12 


Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

Contrast 

Proportion  of 
Variance 

C040801D 

C040801E 

C040801M 

C040802B 

C040802C 

0.89810 

0.89719 

0.88578 

0.89384 

0.88248 

NATLUNCH 
NATLUNCL 
REM  READ 

0.89537 

0.88462 

0.86228 

Appendix  D 

IRT  PARAMETERS 


This  appendix  contains  tables  of  IRT  (item  response  theory)  parameters  for  NAEP  items 
that  were  scaled  in  each  subject  area  and  study  (main  and  long-term  trend)  for  which  IRT  scales 
were  created. 

For  each  of  the  binary  scored  items  used  in  scaling  (i.e.,  multiple-choice  items  and  short 
constructed-response  items),  the  tables  provide  estimates  of  the  IRT  parameters  (which 
correspond  to  aj,  bj,  and  cj  in  Equation  11.1  in  Chapter  11)  and  their  associated  standard  errors 
(s.e.)  of  the  estimates.  For  each  of  the  polytomously  scored  items  (i.e.,  the  extended  constructed- 
response  items),  the  tables  also  show  the  estimates  of  the  djy  parameters  (see  Equation  1 1.3)  and 
their  associated  standard  errors. 

For  the  main  assessment  items,  the  tables  also  show  the  block  in  which  each  item  appears 
for  each  age  class  (Block)  and  the  position  of  each  item  within  its  block  (Item). 

Note  that  item  parameters  shown  in  this  appendix  are  in  the  metrics  used  for  the  original 
calibration  of  the  scales.  The  transformations  needed  to  represent  these  parameters  in  terms  of 
the  metric  of  the  final  reporting  scales  are  given  in  Chapters  12  through  18. 


ERIC 


722 


671 


Table  D-1 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Number  Sense,  Properties,  and  Operations,  Grade  4 
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Table  D-1  (continued) 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Number  Sense,  Properties,  and  Operations,  Grade  4 
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Table  D-2 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Measurement,  Grade  4 
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Table  D-3 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Geometry  and  Spatial  Sense,  Grade  4 
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Table  D-4 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Data  Analysis,  Statistics,  and  Probabilties,  Grade  4 
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Table  D-5 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Algebra  and  Functions,  Grade  4 
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Table  D-6 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Number  Sense,  Properties,  and  Operations,  Grade  8 
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Table  D-6  (continued) 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Number  Sense,  Properties,  and  Operations,  Grade  8 
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Table  D-7 

IRT  Parameters  for  the  Mathematics  Main  Samples 
Measurement,  Grade  8 
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Table  D-8 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Geometry  and  Spatial  Sense,  Grade  8 
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Table  D-9 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Data  Analysis,  Statistics,  and  Probability,  Grade  8 
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Table  D-10 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Algebra  and  Functions,  Grade  8 
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Table  D-11 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Number  Sense,  Properties,  and  Operations,  Grade  12 
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Table  D-12 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Measurement,  Grade  12 
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Table  D-13 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Geometry  and  Spatial  Sense,  Grade  12 
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Table  D-14 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Data  Analysis,  Statistics,  and  Probability,  Grade  12 
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Table  D-15 

IRT  Parameters  for  the  1996  Mathematics  Main  Samples 
Algebra  and  Functions,  Grade  12 
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Table  D-16 

IRT  Parameters  for  the  1996  Science  Main  Samples 
Physical  Science,  Grade  4 
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Table  D-17 

IRT  Parameters  for  the  1996  Science  Main  Samples 
Earth  Science,  Grade  4 
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Table  D-17  (continued) 

IRT  Parameters  for  the  1996  Science  Main  Samples 
Earth  Science,  Grade  4 
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Table  D-18 

IRT  Parameters  for  the  1996  Science  Main  Samples 
Life  Science,  Grade  4 
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IRT  Parameters  for  the  1996  Science  Main  Samples 
Physical  Science,  Grade  8 
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IRT  Parameters  for  the  1996  Science  Main  Samples 
Earth  Science,  Grade  8 
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Table  D-20  (continued) 

IRT  Parameters  for  the  1996  Science  Main  Samples 
Earth  Science,  Grade  8 
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IRT  Parameters  for  the  1996  Science  Main  Samples 
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Table  D-23 

IRT  Parameters  for  1996  Science  Items 
Earth  Science,  Grade  12 
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IRT  Parameters  for  Science  Items 
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Life  Science,  Grade  12 
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Table  D-25 

IRT  Parameters  for  the  Reading  Long-Term  Trend  Samples,  Age  9 


NAEPID 

A- 

S.E. 

B 

S.E. 

C 

NOOllOl 

0.594 

(0.152) 

1.629 

(0.236) 

0.358 

N008601 

1.728 

(0.142) 

-0.227 

(0.053) 

0.245 

N008602 

1.362 

(0.119) 

0.124 

(0.053) 

0.218 

N008603 

1.226 

(0.105) 

-0.259 

(0.070) 

0.223 

N008701 

0.548 

(0.056) 

-3.106 

(0.311) 

0.281 

N001501 

2.488 

(0.243) 

-0.508 

(0.055) 

0.329 

N001502 

2.331 

(0.191) 

0.319 

(0.031) 

0.222 

N001503 

2.140 

(0.192) 

-0.080 

(0.047) 

0.329 

N001504 

2.368 

(0.204) 

0.262 

(0.034) 

0.279 

N001507 

1.043 

(0.138) 

2.112 

(0.194) 

0.000 

N001601 

1.213 

(0.123) 

0.186 

(0.065) 

0.292 

N001602 

1.934 

(0.193) 

0.470 

(0.040) 

0.304 

N001603 

1.335 

(0.201) 

0.929 

(0.062) 

0.319 

N001604 

1.505 

(0.176) 

0.775 

(0.045) 

0.233 

N008801 

1.461 

(0.137) 

-0.839 

(0.093) 

0.318 

N001801 

0.334 

(0.097) 

6.427 

(1.925) 

0.144 

N001802 

2.334 

(0.248) 

1.461 

(0.070) 

0.203 

N008901 

1.835 

(0.160) 

-0.140 

(0.051) 

0.276 

N008902 

1.281 

(0.123) 

-0.204 

(0.075) 

0.304 

N008905 

0.156 

(0.000) 

15.358 

(0.000) 

0.000 

N002001 

2.113 

(0.173) 

0.884 

(0.033) 

0.197 

N002002 

1.582 

(0.166) 

0.675 

(0.041) 

0.220 

N002003 

1.619 

(0.176) 

0.568 

(0.045) 

0.289 

N009001 

1.762 

(0.162) 

0.518 

(0.036) 

0.190 

N009002 

1.645 

(0.175) 

0.679 

(0.041) 

0.221 

N009003 

1.659 

(0.191) 

1.105 

(0.055) 

0.240 

N009004 

2.304 

(0.233) 

0.471 

(0.036) 

0.313 

N009101 

0.933 

(0.095) 

-0.722 

(0.132) 

0.286 

N009201 

1.480 

(0.145) 

-0.594 

(0.086) 

0.330 

N002101 

1.519 

(0.259) 

1.569 

(0.123) 

0.236 

N002102 

2.003 

(0.298) 

1.679 

(0.116) 

0.163 

N002702 

1.702 

(0.190) 

0.809 

(0.039) 

0.190 

N009601 

0.863 

(0.085) 

-1.126 

(0.165) 

0.272 

N002401 

1.747 

(0.170) 

0.730 

(0.035) 

0.146 

N009401 

1.920 

(0.162) 

-0.257 

(0.050) 

0.245 

N002801 

3.165 

(0.260) 

0.261 

(0.025) 

0.179 

N002802 

2.501 

(0.215) 

0.182 

(0.033) 

0.220 

N002804 

0.662 

(0.076) 

1.665 

(0.147) 

0.000 

N009701 

1.231 

(0.122) 

0.246 

(0.060) 

0.256 

N009702 

1.867 

(0.171) 

0.286 

(0.042) 

0.273 

N009703 

2.056 

(0.205) 

0.673 

(0.035) 

0.274 

N009704 

1.936 

(0.194) 

0.683 

(0.034) 

. 0.201 

N009705 

1.755 

(0.155) 

0.193 

(0.044) 

0.264 

N003001 

0.970 

(0.257) 

1.971 

(0.262) 

0.195 

N003002 

0.525 

(0.085) 

0.663 

(0.150) 

0.219 

N003003 

0.081 

(0.018) 

14.651 

(3.865) 

0.032 

NO 14001 

1.414 

(0.138) 

0.083 

(0.060) 

0.283 

N003101 

1.392 

(0.146) 

0.149 

(0.062) 

0.300 

N003102 

3.430 

(0.222) 

0.686 

(0.023) 

0.199 

N003104 

0.869 

(0.138) 

2.355 

(0.280) 

0.000 

N009801 

1.224 

(0.121) 

-1.584 

(0.151) 

0.337 

N009901 

1.091 

(0.115) 

0.043 

(0.079) 

0.296 

NO  14301 

3.051 

(0.257) 

0.310 

(0.026) 

0.243 

NO 14302 

1.658 

(0.161) 

0.464 

(0.042) 

0.248 

NO  14303 

2.697 

(0.229) 

0.067 

(0.033) 

0.271 

S.E. 

(0.035) 

(0.027) 

(0.026) 

(0.029) 

(0.065) 

(0.029) 

(0.021) 

(0.027) 

(0.023) 

(0.000) 

(0.028) 

(0.022) 

(0.023) 

(0.021) 

(0.036) 

(0.014) 

(0.012) 

(0.027) 

(0.031) 

(0.000) 

(0.015) 

(0.021) 

(0.023) 

(0.020) 

(0.021) 

(0.017) 

(0.022) 

(0.042) 

(0.034) 

(0.016) 

(0.012) 

(0.019) 

(0.052) 

(0.017) 

(0.027) 

(0.019) 

(0.023) 

(0.000) 

(0.027) 

(0.024) 

(0.019) 

(0.018) 

(0.024) 

(0.019) 

(0.041) 

(0.010) 

(0.028) 

(0.028) 

(0.015) 

(0.000) 

(0.054) 

(0.032) 

(0.019) 

(0.023) 

(0.023) 
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Table  D-25  (continued) 

IRT  Parameters  for  the  Reading  Long-Term  Trend  Samples,  Age  9 


NAEPED 

A 

S.E. 

B 

S.E. 

C 

S.E. 

NO 10002 

1.783 

(0.173) 

0.191 

(0.049) 

0.326 

(0.026) 

NO 10003 

1.940 

(0.165) 

0.210 

(0.039) 

0.234 

(0.023) 

N010102 

1.881 

(0.219) 

0.661 

(0.042) 

0.292 

(0.022) 

N010103 

2.171 

(0.188) 

0.072 

(0.040) 

0.261 

(0.025) 

N003701 

1.266 

(0.135) 

-0.042 

(0.078) 

0.327 

(0.032) 

N003702 

1.990 

(0.190) 

0.403 

(0.038) 

0.240 

(0.023) 

N003704 

0.731 

(0.073) 

1.196 

(0.091) 

0.000 

(0.000) 

N003801 

1.287 

(0.395) 

1.892 

(0.267) 

0.345 

(0.018) 

N003802 

0.511 

(0.086) 

0.499 

(0.176) 

0.259 

(0.046) 

N003803 

1.130 

(0.341) 

2.034 

(0.299) 

0.228 

(0.017) 

N010301 

0.656 

(0.070) 

-1.167 

(0.215) 

0.291 

(0.057) 

NO 10201 

0.954 

(0.092) 

-1.367 

(0.166) 

0.303 

(0.053) 

N004101 

1.292 

(0.127) 

-0.134 

(0.075) 

0.316 

(0.032) 

N004201 

1.546 

(0.205) 

0.926 

(0.052) 

0.263 

(0.021) 

N004202 

1.113 

(0.190) 

1.015 

(0.082) 

0.314 

(0.026) 

N010401 

0.703 

(0.077) 

-0.817 

(0.175) 

0.262 

(0.049) 

NO 10402 

1.642 

(0.207) 

0.899 

(0.049) 

0.255 

(0.020) 

NO 10403 

1.612 

(0.205) 

1.119 

(0.061) 

0.233 

(0.018) 

N004701 

2.232 

(0.185) 

0.400 

(0.030) 

0.190 

(0.019) 

N004702 

1.016 

(0.121) 

0.167 

(0.089) 

0.339 

(0.033) 

N004703 

1.977 

(0.165) 

0.318 

(0.035) 

0.216 

(0.021) 

N004801 

1.232 

(0.123) 

-0.314 

(0.087) 

0.340 

(0.034) 

N004901 

2.143 

(0.185) 

0.980 

(0.039) 

0.250 

(0.015) 

N005101 

0.743 

(0.068) 

-1.734 

(0.202) 

0.281 

(0.060) 

N010801 

1.208 

(0.131) 

0.407 

(0.060) 

0.252 

(0.027) 

NO  10902 

2.336 

(0.232) 

0.531 

(0.033) 

0.258 

(0.020) 

N010903 

2.951 

(0.249) 

0.322 

(0.027) 

0.198 

(0.019) 

N010904 

2.415 

(0.212) 

0.653 

(0.031) 

0.257 

(0.019) 

N014101 

0.980 

(0.108) 

-0.014 

(0.092) 

0.257 

(0.034) 

NOllOOl 

1.653 

(0.112) 

0.277 

(0.034) 

0.293 

(0.018) 

NO  11002 

2.323 

(0.153) 

0.574 

(0.022) 

0.267 

(0.014) 

NO  11003 

2.619 

(0.158) 

0.019 

(0.025) 

0.287 

(0.017) 

NO 11004 

2.453 

(0.145) 

0.341 

(0.021) 

0.228 

(0.015) 

NOlllOl 

2.248 

(0.136) 

0.405 

(0.022) 

0.218 

(0.014) 

N011201 

1.441 

(0.116) 

0.604 

(0.035) 

0.264 

(0.017) 

N011301 

2.159 

(0.137) 

0.292 

(0.026) 

0.279 

(0.016) 

NO 11302 

1.528 

(0.142) 

0.647 

(0.038)  . 

0.347 

(0.018) 

N011401 

2.891 

(0.180) 

1.102 

(0.032) 

0.392 

(0.011) 

N011402 

1.010 

(0.126) 

0.944 

(0.061) 

0.297 

(0.021) 

N011403 

2.055 

(0.156) 

1.132 

(0.039) 

0.297 

(0.011) 

N011404 

1.869 

(0.139) 

1.051 

(0.034)- 

0.214 

(0.012) 

NO 13201 

2.445 

(0.196) 

0.203 

(0.031) 

0.234 

(0.021) 

N013301 

1.657 

(0.164) 

-0.361 

(0.072) 

0.416 

(0.030) 

N013401 

1.676 

(0.154) 

0.581 

(0.036) 

0.183 

(0.019) 

NO 13402 

2.405 

(0.225) 

0.240 

(0.037) 

0.355 

(0.023) 

NO 13403 

2.556 

(0.219) 

0.553 

(0.027) 

0.234 

(0.017) 

N014201 

1.325 

(0.130) 

0.042 

(0.065) 

0.334 

(0.028) 

N014501 

0.687 

(0.041) 

-0.625 

(0.062) 

0.000 

(0.000) 

NO 14502 

0.635 

(0.036) 

-0.341 

(0.065) 

0.000 

(0.000) 

NO 14502 

0.718 

(0.058) 

-0.738 

(0.088) 

0.000 

(0.000) 

N014503 

0.824 

(0.045) 

-1.253 

(0.071) 

0.000 

(0.000) 
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NAEPID 


NOOllOl 

N001201 

N001202 

N001301 

N001302 

N001303 

N001401 

N001501 

N001502 

N001503 

N001504 

N001507 

N001601 

N001602 

N001603 

N001604 

N001701 

N001702 

N001703 

N001801 

N001802 

N001901 

N001904 

N002001 

N002002 

N002003 

N002101 

N002102 

N002201 

N002202 

N002203 

N002401 

N002501 

N002701 

N002801 

N002802 

N002804 

N002902 

N002903 

N002904 

N002905 

N002906 

N003001 

N003002 

N003003 

N003101 

N003102 

N003104 

N003201 

N003202 

N003203 

N003204 

N003301 

N003401 

N003501 


Table  D-26 

IRT  Parameters  for  the  Reading  Long-Term  Trend  Samples,  Age  13 


A 

S.E. 

B 

0.221 

(0.038) 

0.647 

0.537 

(0.108) 

1.467 

1.548 

(0.162) 

0.812 

0.633 

(0.091) 

0.151 

0.598 

(0.075) 

-2.213 

0.894 

(0.100) 

0.588 

0.919 

(0.087) 

-0.173 

2.067 

(0.183) 

-1.655 

1.414 

(0.108) 

-0.684 

1.124 

(0.089) 

-1.196 

1.260 

(0.098) 

-0.746 

0.536 

(0.048) 

2.054 

0.336 

(0.044) 

-1.237 

0.865 

(0.076) 

-1.473 

0.749 

(0.091) 

-0.010 

0.766 

(0.069) 

-0.560 

0.692 

(0.066) 

-0.819 

0.807 

(0.187) 

2.399 

0.693 

(0.067) 

-0.283 

0.128 

(0.035) 

11.783 

1.001 

(0.117) 

0.753 

0.781 

(0.080) 

0.007 

0.805 

(0.000) 

0.242 

0.995 

(0.078) 

-0.167 

1.095 

(0.100) 

-0.150 

0.943 

(0.081) 

-0.674 

0.703 

(0.110) 

1.336 

1.227 

(0.117) 

0.824 

0.958 

(0.000) 

-0.393 

0.980 

(0.099) 

-0.560 

0.483 

(0.049) 

-1.986 

0.800 

(0.060) 

-0.904 

0.533 

(0.059) 

0.026 

0.706 

(0.085) 

0.462 

1.255 

(0.098) 

-1.143 

1.276 

(0.104) 

-1.412 

0.340 

(0.000) 

2.103 

0.564 

(0.056) 

-1.336 

1.325 

(0.112) 

-0.745 

0.891 

(0.081) 

-0.272 

0.570 

(0.070) 

0.426 

1.186 

(0.094) 

-0.829 

0.669 

(0.095) 

1.313 

0.292 

(0.039) 

-0.126 

1.841 

(0.209) 

2.355 

1.080 

(0.091) 

-1.113 

1.489 

(0.121) 

-0.418 

0.569 

(0.047) 

1.758 

0.876 

(0.075) 

-0.919 

0.973 

(0.087) 

0.155 

1.243 

(0.125) 

0.308 

1.049 

(0.103) 

0.408 

0.772 

(0.070) 

-0.573 

0.999 

(0.085) 

-0.251 

0.809 

(0.078) 

S.E. 

C 

S.E. 

(0.397) 

0.290 

(0.048) 

(0.191) 

0.336 

(0.037) 

(0.046) 

0.241 

(0.018) 

(0.192) 

0.433 

(0.045) 

(0.345) 

0.528 

(0.070) 

(0.082) 

0.246 

(0.029) 

(0.100) 

0.273 

(0.038) 

(0.077) 

0.271 

(0.048) 

(0.065) 

0.214 

(0.033) 

(0.101) 

0.233  . 

(0.045) 

(0.077) 

0.226 

(0.036) 

(0.158) 

0.000 

(0.000) 

(0.371) 

0.286 

(0.066) 

(0.160) 

0.286 

(0.060) 

(0.146) 

0.343 

(0.045) 

(0.127) 

0.235 

(0.045) 

(0.162) 

0.265 

(0.052) 

(0.253) 

0.249 

(0.018) 

(0.130) 

0.227 

(0.043) 

(3.143) 

0.050 

(0.013) 

(0.073) 

0.264 

(0.026) 

(0.112) 

0.235 

(0.039) 

(0.000) 

0.000 

(0.000) 

(0.071) 

0.170 

(0.030) 

(0.082) 

0.284 

(0.034) 

(0.108) 

0.254 

(0.043) 

(0.116) 

0.223 

(0.029) 

(0.049) 

0.144 

(0.018) 

(0.061) 

0.195 

(0.029) 

(0.125) 

0.347 

(0.046) 

(0.267) 

0.265 

(0.062) 

(0.108) 

0.153 

(0.041) 

(0.160) 

0.211 

(0.045) 

(0.112) 

0.235 

(0.037) 

(0.089) 

0.217 

(0.042) 

(0.099) 

0.230 

(0.047) 

(0.000) 

0.000 

(0.000) 

(0.222) 

0.274 

(0.061) 

(0.083) 

0.285 

(0.039) 

(0.100) 

0.238 

(0.038) 

(0.145) 

0.223 

(0.041) 

(0.085) 

0.228 

(0.039) 

(0.111) 

0.170 

(0.028) 

(0.291) 

0.179 

(0.052) 

(0.118) 

0.091 

(0.007) 

(0.111) 

0.256 

(0.048) 

(0.062) 

0.258 

(0.031) 

(0.125) 

0.000 

(0.000) 

(0.125) 

0.246 

(0.047) 

(0.073) 

0.194 

(0.029) 

(0.066) 

0.306 

(0.027) 

(0.068) 

0.228 

(0.027) 

(0.126) 

0.233 

(0.045) 

(0.082) 

0.198 

(0.034) 

(0.131) 

0.268 

(0.046) 

Table  D-26  (continued) 

IRT  Parameters  for  the  Reading  Long-Term  Trend  Samples,  Age  13 


NAEPID 

A 

S.E. 

B 

S.E. 

c 

S.E. 

N003601 

0.934 

(0.081) 

-1.243 

(0.134) 

0.261 

(0.052) 

N003602 

0.977 

(0.082) 

-0.350 

(0.086) 

0.205 

(0.035) 

N003701 

0.814 

(0.073) 

-0.885 

(0.135) 

0.245 

(0.049) 

N003702 

1.107 

(0.112) 

-0.107 

(0.089) 

0.315 

(0.036) 

N003704 

0.596 

(0.043) 

0.105 

(0.059) 

0.000 

(0.000) 

N003801 

0.329 

(0.059) 

1.324 

(0.279) 

0.199 

(0.051) 

N003802 

0.248 

(0.035) 

-1.477 

(0.413) 

0.198 

(0.059) 

N003803 

0.419 

(0.113) 

3.034 

(0.489) 

0.243 

(0.031) 

N004002 

0.508 

(0.051) 

-2.289 

(0.268) 

0.265 

(0.063) 

N003901 

1.095 

(0.112) 

-2.541 

(0.162) 

0.256 

(0.061) 

N004101 

0.947 

(0.081) 

-1.588 

(0.145) 

0.272 

(0.058) 

N004201 

0.865 

(0.081) 

-0.079 

(0.097) 

0.237 

(0.037) 

N004202 

0.638 

(0.074) 

-0.078 

(0.158) 

0.274 

(0.047) 

N004301 

1.058 

(0.106) 

0.288 

(0.075) 

0.276 

(0.030) 

N004303 

0.859 

(0.051) 

0.222 

(0.041) 

0.000 

(0.000) 

N004401 

1.505 

(0.146) 

-2.244 

(0.116) 

0.265 

(0.060) 

N004402 

0.810 

(0.073) 

-0.247 

(0.106) 

0.215 

(0.039) 

N004403 

1.269 

(0.110) 

-1.673 

(0.116) 

0.268 

(0.055) 

N004501 

0.600 

(0.083) 

0.307 

(0.170) 

0.316 

(0.046) 

N004502 

0.628 

(0.059) 

-0.957 

(0.174) 

0.250 

(0.053) 

N004601 

0.844 

(0.082) 

0.224 

(0.087) 

0.204 

(0.032) 

N004602 

1.020 

(0.087) 

-0.184 

(0.080) 

0.231 

(0.033) 

N004603 

1.279 

(0.104) 

-0.539 

(0.074) 

0.263 

(0.035) 

N004605 

0.655 

(0.042) 

-1.096 

(0.078) 

0.000 

(0.000) 

N004701 

1.585 

(0.118) 

-0.800 

(0.059) 

0.187 

(0.031) 

N004702 

0.752 

(0.062) 

-1.296 

(0.144) 

0.228 

(0.051) 

N004703 

0.853 

(0.063) 

-1.176 

(0.108) 

0.183 

(0.041) 

N004801 

1.131 

(0.093) 

-1.351 

(0.112) 

0.250 

(0.049) 

N004901 

0.795 

(0.077) 

-0.077 

(0.107) 

0.239 

(0.038) 

N005101 

0.697 

(0.066) 

-2.475 

(0.220) 

0.265 

(0.063) 

N005001 

1.312 

(0.310) 

2.234 

(0.221) 

0.259 

(0.016) 

N005002 

0.664 

(0.155) 

2.083 

(0.233) 

0.331 

(0.026) 

N005003 

0.950 

(0.174) 

2.096 

(0.163) 

0.170 

(0.016) 

N005201 

0.676 

(0.164) 

1.391 

(0.212) 

0.578 

(0.030) 

N005202 

0.459 

(0.065) 

0.432 

(0.205) 

0.245 

(0.050) 

N005203 

1.074 

(0.232) 

1.989 

(0.166) 

0.329 

(0.017) 

N005301 

0.903 

(0.091) 

-0.092 

(0.100) 

0.265 

(0.037) 

N005302 

1.792 

(0.172) 

0.630 

(0.039) 

0.190 

(0.018) 

N005303 

0.860 

(0.129) 

0.966 

(0.101) 

0.294 

(0.029) 

N005304 

1.766 

(0.162) 

0.185 

(0.046) 

0.236 

(0.023) 

N005305 

0.989 

(0.099) 

-0.716 

(0.123) 

0.293 

(0.047) 

N005403 

1.119 

(0.102) 

-0.629 

(0.102) 

0.330 

(0.042) 

N005404 

1.101 

(0.098) 

-1.432 

(0.131) 

0.293 

(0.056) 

N005405 

1.633 

(0.143) 

0.092 

(0.051) 

0.299 

(0.025) 

N005406 

0.902 

(0.081) 

-0.312 

(0.101) 

0.248 

(0.039) 

N005407 

1.339 

(0.110) 

-0.483 

(0.071) 

0.271 

(0.034) 

N005503 

0.832 

(0.100) 

0.425 

(0.104) 

0.308 

(0.035) 

N005504 

1.157 

(0.129) 

0.998 

(0.059) 

0.201 

(0.019) 

N005505 

0.952 

(0.092) 

-0.874 

(0.139) 

0.334 

(0.052) 

N005601 

1.405 

(0.128) 

-0.519 

(0.079) 

0.350 

(0.036) 

N005602 

1.352 

(0.129) 

0.536 

(0.052) 

0.237 

(0.022) 

N005603 

1.475 

(0.132) 

-0.385 

(0.071) 

0.339 

(0.033) 

722 
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Table  D-27 

IRT  Parameters  for  the  Reading  Long-Term  Trend  Samples,  Age  17 


NAEPID 

A 

S.E. 

B 

S.E. 

c 

NO 17001 

1.474 

(0.141) 

0.014 

(0.063) 

0.331 

NO 17002 

1.717 

(0.160) 

0.465 

(0.044) 

0.239 

NO 17003 

1.698 

(0.170) 

1.348 

(0.059) 

0.230 

N001301 

1.071 

(0.147) 

-0.065 

(0.133) 

0.569 

N001302 

0.663 

(0.093) 

-2.474 

(0.384) 

0.592 

N001303 

0.910 

(0.089) 

-0.237 

(0.103) 

0.268 

N001401 

0.709 

(0.076) 

-1.101 

(0.196) 

0.318 

N001501 

1.527 

(0.165) 

-2.116 

(0.118) 

0.261 

N001502 

1.643 

(0.139) 

-1.079 

(0.070) 

0.194 

N001503 

1.173 

(0.108) 

-1.675 

(0.129) 

0.265 

N001504 

1.339 

(0.117) 

-1.155 

(0.091) 

0.239 

N001507 

0.358 

(0.042) 

2.182 

(0.241) 

0.000 

N001701 

0.621 

(0.064) 

-1.558 

(0.223) 

0.294 

N001702 

0.588 

(0.169) 

3.082 

(0.514) 

0.308 

N001703 

0.940 

(0.098) 

-0.612 

(0.131) 

0.348 

N001901 

0.981 

(0.093) 

-0.927 

(0.124) 

0.288 

N001904 

0.748 

(0.049) 

-1.258 

(0.081) 

0.000 

N002001 

1.268 

(0.111) 

-0.471 

(0.075) 

0.250 

N002002 

0.983 

(0.087) 

-0.794 

(0.105) 

0.231 

N002003 

1.183 

(0.113) 

-1.174 

(0.116) 

0.294 

N002101 

0.614 

(0.070) 

0.190 

(0.129) 

0.173 

N002102 

1.538 

(0.130) 

0.113 

(0.049) 

0.197 

N002201 

1.181 

(0.000) 

-0.908 

(0.077) 

0.389 

N002202 

1.657 

(0.202) 

-0.718 

(0.099) 

0.516 

N002203 

0.431 

(0.057) 

-3.349 

(0.441) 

0.291 

N002501 

0.451 

(0.064) 

-0.441 

(0.288) 

0.328 

N002701 

0.736 

(0.070) 

-0.369 

(0.118) 

0.185 

N002702 

0.819 

(0.072) 

-0.973 

(0.129) 

0.194 

N002801 

1.488 

(0.150) 

-1.764 

(0.115) 

0.272 

N002802 

1.152 

(0.115) 

-2.047 

(0.154) 

0.291 

N002804 

0.202 

(0.033) 

2.784 

(0.450) 

0.000 

N002902 

0.648 

(0.071) 

-1.386 

(0.227) 

0.320 

N002903 

1.604 

(0.162) 

-1.199 

(0.095) 

0.305 

N002904 

1.136 

(0.103) 

-0.828 

(0.100) 

0.267 

N002905 

0.658 

(0.079) 

0.118 

(0.141) 

0.249 

N002906 

1.747 

(0.181) 

-1.081 

(0.087) 

0.327 

N003001 

1.000 

(0.095) 

0.414 

(0.066) 

0.159 

N003002 

0.301 

(0.041) 

-0.708 

(0.312) 

0.185 

N003003 

1.528 

(0.154) 

1.291 

(0.052) 

0.089 

N003101 

0.943 

(0.090) 

-1.727 

(0.162) 

0.284 

N003102 

1.270 

(0.117) 

-1.172 

(0.104) 

0.254 

N003104 

0.754 

(0.054) 

1.057 

(0.068) 

0.000 

N003201 

0.934 

(0.092) 

-1.821 

(0.175) 

0.298 

N003202 

1.172 

(0.117) 

-0.722 

(0.106) 

0.345 

N003203 

0.910 

(0.082) 

-0.615 

(0.107) 

0.225 

N003204 

1.069 

(0.095) 

-0.988 

(0.107) 

0.244 

N003301 

1.153 

(0.100) 

-1.165 

(0.103) 

0.224 

NO 15201 

0.707 

(0.070) 

-2.188 

(0.215) 

0.274 

N003501 

0.688 

(0.067) 

-1.041 

(0.163) 

0.248 

N003601 

1.087 

(0.105) 

-1.839 

(0.150) 

0.275 

N003602 

1.226 

(0.114) 

-0.815 

(0.095) 

0.270 

N003701 

0.663 

(0.073) 

-1.445 

(0.226) 

0.315 

N003702 

1.573 

(0.151) 

-0.535 

(0.072) 

0.326 

N003704 

0.664 

(0.047) 

-0.441 

(0.063) 

0.000 

N003801 

0.570 

(0.084) 

0.576 

(0.156) 

0.228 

O "10 


S.E. 


(0.029) 
(0.021) 
(0.014) 
(0.036) 
(0.068) 
(0.039) 
(0.058) 
(0.053) 
(0.034) 
(0.052) 
(0.040) 
(0.000) 
(0.061) 
(0.022) 
(0.046) 
(0.047) 
(0.000) 
(0.035) 
(0.042) 
(0.047) 
(0.041) 
(0.025) 
(0.041) 
(0.040) 
‘ (0.065) 
(0.062) 
(0.042) 
(0.047) 
(0.052) 
(0.059) 
(0.000) 
(0.063) 
(0.044) 
(0.042) 
(0.043) 
(0.042) 
(0.027) 
(0.055) 
(0.011) 
(0.057) 
(0.045) 
(0.000) 
(0.060) 
(0.043) 
(0.041) 
(0.043) 
(0.043) 
(0.060) 
(0.051) 
(0.056) 
(0.042) 
(0.063) 
(0.036) 
(0.000) 
(0.045) 
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Table  D-27  (continued) 

IRT  Parameters  for  the  Reading  Long-Term  Trend  Samples,  Age  17 


NAEPID 

A 

S.E. 

B 

S.E. 

C 

S.E. 

N003802 

0.184 

(0.033) 

-2.639 

(0.654) 

0.199 

(0.059) 

N003803 

0.652 

(0.155) 

1.842 

(0.217) 

0.313 

(0.031) 

NO 16001 

0.978 

(0.095) 

-0.877 

(0.124) 

0.285 

(0.047) 

NO 16002 

0.885 

(0.119) 

0.495 

(0.109) 

0.340 

(0.036) 

NO 16003 

0.870 

(0.093) 

-0.109 

(0.111) 

0.284 

(0.040) 

NO 16004 

1.309 

(0.126) 

-0.428 

(0.082) 

0.322 

(0.037) 

NO 16005 

1.416 

(0.128) 

-0.410 

(0.071) 

0.275 

(0.035) 

NO 16006 

0.881 

(0.086) 

-0.088 

(0.096) 

0.224 

(0.037) 

N004201 

0.895 

(0.093) 

-0.477 

(0.123) 

0.288 

(0.045) 

N004202 

0.685 

(0.090) 

-0.043 

(0.164) 

0.319 

(0.048) 

N004301 

0.891 

(0.096) 

-0.339 

(0.119) 

0.270 

(0.043) 

N004303 

0.673 

(0.054) 

-0.092 

(0.065) 

0.000 

(0.000) 

N015502 

1.250 

(0.110) 

-0.310 

(0.074) 

0.275 

(0.034) 

NO 15503 

1.218 

(0.113) 

0.128 

(0.066) 

0.257 

(0.029) 

NO 15504 

1.220 

(0.107) 

-0.403 

(0.078)  ■ 

. 0.268 

(0.035) 

NO  15505 

0.654 

(0.064) 

-0.944 

(0.167) 

0.252 

(0.050) 

N004501 

0.668 

(0.072) 

-0.631 

(0.168) 

0.280 

(0.051) 

N004502 

0.493 

(0.054) 

-2.025 

(0.285) 

0.288 

(0.063) 

N004601 

0.826 

(0.077) 

-0.149 

(0.097) 

0.206 

(0.036) 

N004602 

1.502 

(0.129) 

-0.553 

- (0.066) 

0.247 

(0.034) 

N004603 

1.335 

(0.121) 

-0.883 

(0.089) 

0.273 

(0.040). 

N004605 

0.749 

(0.054) 

-1.265 

(0.093) 

0.000 

(0.000) 

N005001 

1.980 

(0.209) 

0.743 

(0.041) 

0.251 

(0.018) 

N005002 

0.812 

(0.100) 

0.613 

(0.098) 

0.252 

(0.033) 

N005003 

0.895 

(0.120) 

1.264 

(0.085) 

0.150 

(0.022) 

N004901 

0.850 

(0.094). 

-0.552 

(0.147) 

0.359 

(0.049) 

N005201 

0.769 

(0.158) 

0.585 

(0.202) 

0.635 

(0.035) 

N005202 

0.533 

(0.077) 

0.327 

(0.189) 

0.284 

(0.049) 

N005203 

0.757 

(0.142) 

1.296 

(0.134) 

0.338 

(0.031) 

NO 15901 

1.270 

(0.135) 

0.118 

(0.078) 

0.363 

(0.032) 

NO 15902 

1.198 

(0.123) 

0.205 

(0.074) 

0.299 

(0.031) 

NO 15903 

2.076 

(0.196) 

0.518 

(0.037) 

0.201 

(0.019) 

NO 15905 

0.728 

(0.000) 

0.247 

(0.000) 

0.000 

(0.000) 

N005503 

0.742 

(0.094) 

0.032 

(0.146) 

0.350 

(0.044) 

N005504 

1.381 

(0.137) 

0.345 

(0.060) 

0.294 

(0.026) 

N005505 

0.786 

(0.078) 

-1.926 

(0.207) 

0.309 

(0.064) 

N015101 

1.022 

(0.117) 

0.150 

(0.097) 

0.369 

(0.035) 

N015102 

2.671 

(0.226) 

0.004 

(0.032) 

0.227 

(0.021) 

•N015103 

2.693 

(0.229) 

0.110 

(0.030) 

0.210 

(0.020) 

N015104 

2.168 

(0.191) 

-0.027 

(0.041) 

0.275 

(0.025) 
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Table  D-28 

IRT  Parameters  for  the  Mathematics  Long-Term  Trend  Samples,  Age  9 


NAEPID  A 


N2709011 

0.59950 

N2774011 

0.91059 

N2676011 

1.16597 

N276801 1 

0.69953 

N2768021 

0.58482 

N2768031 

0.54214 

N2507011 

0.64744 

N2507021 

1.13106 

N2507031 

1.01455 

N2622011 

0.72079 

N2572011 

0.93291 

N2761011 

1.03200 

N2861011 

0.88563 

N2700011 

0.57662 

N2721021 

0.81734 

N284001 1 

0.79395 

N2840021 

0.75296 

N2676021 

1.00823 

N262501 1 

0.46201 

N2625021 

0.00000 

N265401 1 

0.47952 

N2661011. 

0.57237 

N2691011 

0.50012 

N268201 1 

0.95788 

N2521011 

0.68677 

N252601 1 

0.00000 

N2723012 

0.77379 

N2766012 

1.16928 

N2578012 

0.64050 

N2634012 

0.77485 

N2634022 

0.90833 

N2735012 

0.62232 

N2754012 

0.98503 

N2775012 

0.77063 

N2776012 

0.84989 

N2776022 

0.83584 

N2776032 

0.79308 

N2614012 

0.41818 

N2506012 

0.93796 

N2506022 

0.57587 

N2506032 

0.95856 

N2514012 

0.70587 

N2509012 

0.48879 

N2509022 

1.09953 

N2509032 

1.18622 

N2503012 

0.72341 

N2760012 

0.95855 

N2760022 

0.83677 

N2861022 

0.90443 

N2711012 

0.73684 

N2520012 

1.13023 

N2690012 

0.59892 

N2768213 

0.00000 

N2768223 

0.00000 

S.E. 

B 

0.03694 

-2.84371 

0.05279 

-1.73828 

0.06386 

-0.69400 

0.04740 

-3.21047 

0.03237 

-2.33512 

0.02503 

-0.05625 

0.03759 

-1.41456 

0.07160 

0.48798 

0.05179 

-0.49314 

0.05486 

-0.72016 

0.05708' 

-0.71528 

0.03885 

-1.02631 

0.03357 

-0.87547 

0.02578 

-0.66154 

0.04941 

-0.47932 

0.03102 

-0.84005 

0.04204 

1.87866 

0.05478 

-0.13182 

0.05964 

0.29971 

0.00000 

-0.10734 

0.13882 

3.46971 

0.07755 

1.35985 

0.07249 

1.52585 

0.07476 

0.62152 

0.09376 

1.62183 

0.00000 

-0.10734 

0.05260 

-2.44300 

0.06842 

-1.14011 

0.04461 

-1.39460 

0.05443 

-1.25946 

0.06814 

-0.42377 

0.05145 

-0.93564 

0.03665 

-0.85998 

0.03025 

-0.68815 

0.03313 

-0.91905 

0.03107 

-0.03557 

0.03006 

-0.21771 

0.03423 

-0.73342 

0.05858 

-1.97101 

0.03736 

-1.88859 

0.05149 

-0.25672 

0.04081 

-0.80890 

0.03297 

-1.81552 

0.06793 

0.42731 

0.06456 

-0.06436 

0.08481 

1.05275 

0.03648 

-0.95870 

0.03543 

1.01588 

0.03271 

0.01609 

0.02885 

-0.23597 

0.12554 

1.77819 

0.10854 

2.99976 

0.00000 

-0.10734 

0.00000 

-0.10734 

S.E. 

c 

0.14206 

0.00000 

0.09895 

0.17174 

0.06026 

0.22100 

0.16225 

0.00000 

0.10902 

0.00000 

0.03920 

■ 0.00000 

0.11571 

0.14224 

0.03863 

0.15349 

0.05496 

0.11757 

0.13738 

0.31391 

0.08508 

0.24537 

0.03378 

0.00000 

0.03512 

0.00000 

0.04489 

0.00000 

0.08147 

0.17124 

0.03760 

0.00000 

0.07965 

0.00000 

0.05033 

0.15327 

0.21852 

0.34233 

0.00000 

0.00000 

0.57466 

0.28410 

0.10758 

0.24159 

0.12337 

0.19987 

0.05230 

0.21573 

0.09947 

0.23094 

0.00000 

0.00000 

0.14599 

0.17087 

0.07632 

0.22568 

0.16037 

0.26842 

0.14323 

0.30576 

0.10049 

0.32971 

0.18471 

0.32768 

0.03191 

0.00000 

0.03564 

0.00000 

0.03691 

0.00000 

0.02749 

0.00000 

0.02949 

0.00000 

0.19602 

0.21652 

0.10891 

0.17798 

0.15604 

0.17040 

0.05505 

0.12352 

0.09514 

0.14658 

0.17006 

0.15974 

0.04018 

0.14315 

0.04272 

0.13661 

0.08474 

0.29008 

0.03430 

0.00000 

0.03830 

0.00000 

0.02577 

0.00000 

0.03139 

0.00000 

0.07829 

0.22868 

0.29445 

0.08071 

0.00000 

0.00000 

0.00000 

0.00000 

O .2 

ERIC 


73u 


S.E, 

0.00000 

0.04417 

0.02978 

0.00000 

0.00000 

0.00000 

0.04151 

0.01635 

0.02566 

0.04583 

0.03633 

0.00000 

0.00000 

0.00000 

0.03313 

0.00000 

0.00000 

0.02205 

0.04846 

0.00000 

0.01985 

0.02796 

0.03081 

0.01992 

0.02074 

0.00000 

0.05028 

0.04110 

0.05431 

0.05417 

0.03773 

0.05576 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.04786 

0.04849 

0.05013 

0.02461 

0.03691 

0.04796 

0.01701 

0.02045 

0.02497 

0.00000 

0.00000 

0.00000 

0.00000 

0.01093 

0.01225 

0.00000 

0.00000 


Table  D-28  (continued) 

IRT  Parameters  for  the  Mathematics  Long-Term  Trend  Samples,  Age  9 


NAEPID 

N2768233 

N2840213 

N2840223 

N2760213 

N2760223 

N2776213 

N2776223 

N2776233 

N2682213 

N2728013 

N2670013 

N2721013 

N2624013 

N2585013 


O 


A S.E.  B 


0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.00000 

0.00000 

-0.10734 

0.89213 

0.05271 

-1.60990 

0.94882 

0.05918 

-1.37676 

0.76836 

0.05370 

-0.95164 

0.80162 

0.07726 

0.52062 

0;56937 

0.08584 

1.68734 

S.E.  C 


0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

.0.00000 

0.00000 

0.10228 

0.18525 

0.04618 

0.10407 

0.26707 

0.04720 

0.12834 

0.28977 

0.04748 

0.08431 

0.31390 

0.02743 

0.12176 

0.22902 

0.02593 
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Table  D-29 

IRT  Parameters  for  the  Mathematics  Long-Term  Trend  Samples,  Age  13 


NAEPBD 

A 

S.E. 

N2819011 

1.14166 

0.08519 

N2546011 

0.84076 

0.06375 

N2768011 

0.50147 

0.04175 

N2768021 

0.50356 

0.03968 

N2768031 

0.40041 

0.02485 

N2776011 

0.68977 

0.03973 

N2776021 

0.68283 

0.03180 

N2776031 

0.61362 

0.03160 

N2672011 

1.01261 

0.07773 

N2862011 

0.96660 

0.05721 

N2509011 

0.35161 

0.03194 

N2509021 

0.86649 

0.04567 

N2509031 

0.82089 

0.05250 

N262401 1 

1.15737 

0.07332 

N2748011 

1.36812 

0.11261 

N2652021 

0.71187 

0.05409 

N2668011 

0.68302 

0.04711 

N252901 1 

1.21576 

0.06143 

N262501 1 

0.54813 

0.04319 

N2625021 

0.00000 

0.00000 

N257601 1 

1.26554 

0.04386 

N2652011 

0.67316 

0.05615 

N2739011 

1.58900 

0.08805 

N2588011 

1.18663 

0.10308 

N2631011 

0.64190 

0.02742 

N265901 1 

0.76359 

0.06972 

N2521011 

0.98805 

0.08716 

N275001 1 

0.86228 

0.03310 

N2601011 

1.51040 

0.08872 

N269001 1 

1.18736 

0.06881 

N2863011 

1.30556 

0.07778 

N2546021 

0.94949 

0.08998 

N2610011 

0.68078 

0.05390 

N286501 1 

0.92314 

0.06419 

N2789041 

0.65447 

0.07568 

N2557011 

1.02120 

0.06648 

N2831011 

1.98595 

0.11576 

N2774012 

0.55830 

0.04494 

N2779012 

0.69697 

0.04398 

N2779022 

0.70057 

0.04447 

N2779032 

0.67984 

0.03835 

N2634012 

0.72108 

0.05722 

N2634022 

0.64681 

0.04723 

N2507012 

0.49809 

0.04120 

N2507022 

0.80945 

0.04387 

N2507032 

0.55562 

0.03849 

N2561012 

0.90472 

0.03943 

N2622012 

0.48839 

0.04072 

N2502012 

0.57688 

0.04537 

N2703012 

0.39105 

0.03095 

N2703022 

1.35410 

0.09816 

N2537012 

0.29752 

0.03037 

N2866012 

0.92129 

0.03326 

N2866022 

0.91481 

0.03328 

B S.E.  C 


-2.21322 

0.11766 

0.22337 

-2.06860 

0.15938 

0.32730 

-4.32401 

0.30550 

0.00000 

-4.07409 

0.27292 

0.00000 

-2.04391 

0.12367 

0.00000 

-2.61117 

0.11894 

0.00000 

-1.49505 

0.06217 

0.00000 

-1.91585 

0.08601 

0.00000 

-0.80989 

0.11129 

0.43931 

-0.84701 

0.08097 

0.25278 

-3.57131 

0.34026 

0.18338 

-0.89285 

0.07248 

0.13528 

-2.08095 

0.12354 

0.15468 

-0.47616 

0.06661 

0.30845 

0.27123 

0.05497 

0.45473 

-0.67179 

0.13437 

0.31397 

-1.17426 

0.13837 

0.28919 

-0.09054 

0.03787 

0.12643 

-1.15266 

0.18669 

0.31206 

-0.11006 

0.00000 

0.00000 

-0.58253 

0.02262 

0.00000 

-2.35874 

0.22164 

0.35331 

-0.08173 

0.03413 

0.21782 

0.67807 

0.05399 

0.37648 

-0.56523 

0.03824 

0.00000 

0.94217 

0.06987 

0.22197 

0.46445 

0.07017 

0.36614 

0.69449 

0.03058 

0.00000 

-0.09785 

0.03903 

0.23718 

0.02047 

0.04434 

0.18630 

0.34034 

0.03658 

0.20346 

1.16354 

0.05776 

0.24036 

0.32732 

0.08902 

0.21317 

0.78062 

0.04668 

0.13451 

1.46561 

0.08759 

0.19367 

0.76744 

0.04130 

0.13699 

0.93254 

0.02435 

0.14242 

-3.39203 

0.25203 

0.19018 

-3.06832 

0.14847 

0.00000 

-3.09463 

0.15011 

0.00000 

-2.53712 

0.11356 

0.00000 

-2.71701 

0.20373 

0.28011 

-2.10054 

0.18187 

0.28513 

-3.73340 

0.28311 

0.13432 

-1.36855 

0.08799 

0.14301 

-2.83807 

0.18703 

0.12102 

-1.51745 

0.05110 

0.00000 

-1.69723 

0.24206 

0.32053 

-1.48973 

0.20035 

0.31995 

-2.32292 

0.25011 

0.15751 

1.71969 

0.05047 

0.08018 

-0.93216 

0.26762 

0.31463 

-0.27410 

0.02565 

0.00000 

-0.38613 

0.02677 

0.00000 

ERIC 


S.E. 

0.04755 

0.05643 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.04191 

0.03707 

0.05197 

0.03282 

0.04439 

0.03123 

0.02029 

0.04560 

0.04765 

0.01933 

0.05227 

0.00000 

0.00000 

0.06308 

0.01910 

0.01843 

0.00000 

0.02236 

0.02419 

0.00000 

0.02132 

0.02171 

0.01687 

0.01713 

0.03023 

0.01724 

0.02217 

0.01524 

0.00858 

0.05347 

0.00000 

0.00000 

0.00000 

0.05922 

0.05652 

0.04782 

0.03659 

0.04294 

0.00000 

0.06053 

0.05791 

0.05437 

0.00648 

0.04325 

0.00000 

0.00000 


NAEP  ID 

N2866032 
N2691012 
N2857012 
N2822012 
N27 89022 
N2635012 
N2588022 
N2789012 
N2647012 
N2615012 
N2618012 
N2616012 
N2613012 
N2612012 
N2814012 
N2520012 
N2588032 
N2789032 
N2865022 
N2692012 
N2768213 
N2768223 
N2768233 
N2806213 
N2806223 
N2806233 
N2806243 
N2806253 
N2806263 
N2789213 
N2645213 
N2599213 
N2789233 
N2789223 
N2789253 
N27 89243 
N2753013 
N2822023 
N2661013 
N2540013 
N2699013 
N2565013 
N2659023 
N2568013 


Table  D-29  (continued) 

IRT  Parameters  for  the  Mathematics  Long-Term  Trend  Samples,  Age  13 


A 

S.E. 

B 

S.E. 

c 

S.E. 

1.09784 

0.03926 

0.60353 

0.02417 

0.00000 

0.00000 

0.93222 

0.06083 

-0.26860 

0.07405 

0.24378 

0.03130 

0.86784 

0.07362 

0.12916 

0.08873 

0.35170 

0.03047 

1.08773 

0.07855 

0.48265 

0.05013 

0.27691 

0.01954 

0.92484 

0.08581 

0.91405 

0.06106 

0.27922 

0.02003 

0.92296 

0.04735 

-0.11892 

0.04787 

0.09969 

0.02117 

1.40917 

0.08526 

0.29574 

0.03525 

0.22132 

0.01693 

1.32488 

0.08361 

0.13825 

0.04266 

0.26789 

0.02004 

1.07150 

0.06803 

0.32213 

0.04605 

0.20016 

0.01983 

0.63407 

0.04418 

-0.86457 

0.13403 

0.24133 

0.04502 

0.59496 

0.04898 

-0.12107 

0.12959 

0.25620 

0.03943 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.52460 

0.05706 

1.13531 

0.10523 

0.17028 

0.02948 

0.62164 

0.10539 

2.17167 

0.16058 

0.24185 

0.02005 

0.72502 

0.10315 

2.14540 

0.13231 

0.16744 

0.01496 

0.98749 

0.07661 

0.58809 

0.05498 

0.25292 

0.02083 

1.46701 

0.10537 

0.95767 

0.03386 

0.19433 

0.01194 

1.88578 

0.12786 

0.80740 

0.02750 

0.22164 

0.01115 

1.01183 

0.06869 

1.01892 

0.04089 

0.10150 

0.01280 

0.90312 

0.04330 

1.68434 

0.05771 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.11006 

0.00000 

0.00000 

0.00000 

0.32096 

0.02984 

-2.49410 

0.32055 

0.20248 

0.05623 

1.16158 

0.08450 

-0.31459 

0.07300 

0.37877 

0.03067 

0.83754 

0.05917 

-0.66475 

0.10688 

0.30429 

0.04122 

0.89809 

0.05678 

-0.52647 

0.08167 

0.22012 

0.03549 

0.77154 

0.05560 

-0.43978 

0.10526 

0.27645 

0.03869 

1.28677 

0.09122 

0.32289 

0.04610 

0.29533 

0.01980 

0.76494 

0.09719 

1.39971 

0.08804 

0.30157 

0.02119 

1.31502 

0.09495 

0.46865 

0.04327 

0.28610 

0.01807 

79S 
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Table  D-30 

IRT  Parameters  for  the  Mathematics  Long-Term  Trend  Samples,  Age  17 


NAEPID 


S.E. 


B 


S.E. 


S.E. 


N2561011 

0.79491 

0.04311 

N260601 1 

1.34667 

0.06412 

N2624011 

0.99863 

0.06306 

N25 88041 

0.77466 

0.05655 

N2860011 

0.77406 

0.04238 

N2860021 

0.98452 

0.05540 

N285701 1 

0.00000 

0.00000 

N2863021 

0.93379 

0.05727 

N2785011 

0.89587 

0.03431 

N2785021 

0.91341 

0.03334 

N2785031 

0.73752 

0.03016 

N2588021 

1.61427 

0.09376 

N2546021 

1.23038 

0.06548 

N2599011 

0.88206 

0.05936 

N2871011 

1.09599 

0.06619 

N2703011 

0.79357 

0.05233 

N2703021 

1.23893 

0.05782 

N2557011 

1.20628 

0.06659 

N2543011 

0.94198 

0.06977 

N2865021 

1.35117 

0.06894 

N2609011 

1.67439 

0.08737 

N256801 1 

1.19200 

0.07693 

N2588031 

1.16183 

0.06826 

N2626011 

0.71586 

0.05636 

N2539011 

1.27296 

0.07183 

N2539021 

0.68407 

0.07353 

N2539031 

0.97734 

0.07560 

N2539041 

1.60610 

0.12066 

N263001 1 

0.63484 

0.02811 

N2789051 

0.56862 

0.06637 

N2873011 

0.67899 

0.02795 

N2873021 

0.78625 

0.03117 

N2643011 

0.81348 

0.03464 

N282801 1 

0.00000 

0.00000 

N25 11011 

1.19012 

0.04410 

N2546012 

1.01954 

0.07885 

N2668012 

0.00000 

0.00000 

N2623012 

0.59455 

0.04634 

N2632012 

0.74713 

0.05476 

N2632022 

0.79125 

0.06335 

N2601012 

1.23905 

0.07225 

N2540012 

0.85834 

0.05622 

N2690012 

1.72750 

0.10979 

N2789012 

0.95891 

0.05614 

N2615012 

0.66473 

0.04366 

N2618012 

0.54161 

0.03672 

N2612012 

0.49101 

0.04376 

N2616012 

0.77994 

0.12183 

N2613012 

0.48799 

0.03928 

N28 14012 

0.48840 

0.06089 

N2804012 

0.57974 

0.02681 

N2590012 

1.00534 

0.03568 

N2871022 

0.99647 

0.05467 

N2863012 

0.96169 

0.05614 

-2.20982 

0.08772 

0.00000 

0.00000 

-1.70690 

0.04401 

0.00000 

0.00000 

-1.49109 

0.10305 

0.25117 

0.05003 

-2.23284 

0.16806 

0.26626 

0.06341 

-1.25574 

0.09080  ’ 

0.14979 

0.03848 

-1.72376 

0.08688 

0.13072 

0.04205 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.95066 

0.09019 

0.24177 

0.04096 

-0.75799 

0.03220 

0.00000 

0.00000 

-0.32812 

0.02674 

0.00000 

0.00000 

-0.77553 

0.03796 

0.00000 

0.00000 

-0.47732 

0.04245 

0.24547 

0.02510 

-0.43454 

0.04751 

0.16243 

0.02526 

-0.23800 

0.08061 

0.23007 

0.03306 

-0.59405 

0.06787 

0.24571 

0.03281 

-2.31065 

0.13649 

0.15273 

0.05089 

-0.22615 

0.03493 

0.08123 

0.01757 

-1.06349 

0.06334 

0.19375 

0.03513 

0.02816 

0.07569 

0.27980 

0.02958 

-0.49347 

0.04182 

0.14970 

0.02354 

-0.21545 

0.03166 

0.17280 

0.01842 

-0.48209 

0.06609 

0.30551 

0.03125 

-0.11321 

0.04989 

0.19059 

0.02408 

0.23733 

0.09052 

0.21434 

0.03145 

-0.50724 

0.05195 

0.21597 

0.02767 

0.33672 

0.13250 

0.37471 

0.03661 

0.38247 

0.06450 

0.29387 

0.02356 

0.45194 

0.04055 

0.37398 

0.01608 

0.85767 

0.04419 

0.00000 

0.00000 

1.05364 

0.11226 

0.22750 

0.03187 

0.39321 

0.03417 

0.00000 

0.00000 

0.69872 

0.03406 

0.00000 

0.00000 

1.21058 

0.04413 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

0.91059 

0.02739 

0.00000 

0.00000 

-2.47842 

0.14197 

0.23942 

0.05824 

-0.08216 

0.00000 

0.00000 

0.00000 

-1.38437 

0.19974 

0.28418 

0.06366 

-1.66485 

0.16535 

0.34067 

0.05986 

-0.77153 

0.14160 

0.38347 

0.04787 

-1.39281 

0.07222 

0.20711 

0.04085 

-1.11846 

0.11377 

0.24266 

0.04961 

-0.25555 

0.04099 

0.30837 

0.02264 

-0.73878 

0.07703 

0.20365 

0.03609 

-2.09292 

0.14828 

0.19419 

0.05189 

-1.47640 

0.16169 

0.20938 

0.05067 

0.02825 

0.15874 

0.21235 

0.04344 

1.78405 

0.10980 

0.34878 

0.01845 

0.14487 

0.13050 

0.15626 

0.03714 

1.62588 

0.12025 

0.15176 

0.02779 

-0.93122 

0.05039 

0.00000 

0.00000 

-0.30252 

0.02469 

0.00000 

0.00000 

-0.90878 

0.07107 

0.18005 

0.03498 

-0.97244 

0.08183 

0.21935 

0.03854 

Table  D-30  (continued) 

IRT  Parameters  for  the  Mathematics  Long-Term  Trend  Samples,  Age  17 


NAEP  BO A 


N2865012 

1.20266 

0.07136 

N2625012 

0.52646 

0.04288 

N2625022 

1.16549 

0.11963 

N2631012 

0.70471 

0.02979 

N2588012 

1.30308 

0.08950 

N2647012 

1.22357 

0.06770 

N2610012 

0.79100 

0.05004 

N2517012 

0.83140 

0.05163 

N2789022 

1.01373 

0.06788 

N2608012 

1.38183 

0.04597 

N2789032 

1.22774 

0.07459 

N2556012 

1.88440 

0.12169 

N2553012 

0.00000 

0.00000 

N2689012 

1.65811 

0.09139 

N2688012 

1.29277 

0.08393 

N2558012 

0.78131 

0.03773 

N2768213 

0.00000 

0.00000 

N2768223 

0.00000 

0.00000 

N2768233 

0.00000 

0.00000 

N2806213 

0.00000 

0.00000 

N2806223 

0.00000 

0.00000 

N2806233 

0.00000 

0.00000 

N2806243 

0.00000 

0.00000 

N2806253 

0.00000 

0.00000 

N2806263 

0.00000 

0.00000 

N2789213 

0.00000 

0.00000 

N2599213 

0.00000 

0.00000 

N2645213 

0.00000 

0.00000 

N2789223 

0.00000 

0.00000 

N2853213 

0.00000 

0.00000 

N2789233 

0.00000 

0.00000 

N2789253 

0.00000 

0.00000 

N2789243 

0.00000 

0.00000 

N2643213 

0.00000 

0.00000 

N2679213 

0.00000 

0.00000 

N2665013 

0.84796 

0.05965 

N2713013 

1.18593 

0.07920 

N2555013 

0.84856 

0.06714 

N2560013 

1.06035 

0.03815 

N2571013 

0.55500 

0.10663 

B 

S.E. 

c 

S.E. 

-1.01036 

0.07055 

0.21977 

0.03858 

-1.55908 

0.22589 

0.34264 

0.06068 

1.40031 

0.05547 

0.27011 

0.01300 

-0.90810 

0.04225 

0.00000 

0.00000 

-0.39955 

0.06397 

0.34481 

0.03014 

-0.39538 

0.04980 

0.19564 

0.02577 

-0.46191 

0.08873 

0.20633 

0.03544 

-0.45041 

0.08232 

0.16674 

0.03513 

-0.33627 

0.07507 

0.26713 

0.03280 

0.04373 

0.01893 

0.00000 

0.00000 

-0.17532 

0.05132 

0.23410 

0.02498 

1.47987 

0.04369 

0.34624 

0.00923 

-0.08216 

0.00000 

0.00000 

0.00000 

0.31535 

0.02827 

0.17930 

0.01355 

1.14930 

0.03461 

0.08848 

0.00897 

1.68142 

0.06260 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.08216 

0.00000 

0.00000 

0.00000 

-0.49696 

0.09867 

0.26261 

0.03943 

-0.08152 

0.05709 

0.27857 

0.02573 

0.20441 

0.08097 

0.26643 

0.02942 

-0.23860 

0.02407 

0.00000 

0.00000 

2.24410 

0.18757 

0.28961 

0.02426 

800 


Table  D-31 

IRT  Parameters  for  the  Science  Long-Term  Trend  Samples,  Age  9 


NAEPID 

A 

S.E. 

N400001 1 

0.56311 

0.04528 

N4002011 

0.00000 

0.00000 

N400301 1 

0.73474 

0.07142 

N400401 1 

0.91803 

0.07765 

N4004021 

1.87737 

0.12423 

N4004031 

0.59447 

0.05294 

N4004041 

1.32191 

0.09209 

N4004051 

0.68599 

0.05778 

N400501 1 

0.49172 

0.06112 

N4001011 

1.04274 

0.16668 

N4001021 

0.80641 

0.10779 

N400601 1 

0.81545 

0.06826 

N4007011 

1.17303 

0.08376 

N400901 1 

0.28543 

0.05136 

N4010011 

0.54419 

0.05162 

N4011011 

0.31592 

0.06357 

N4012011 

0.86721 

0.14000 

N4013011 

0.50247 

0.06336 

N4015012 

0.60132 

0.11386 

N4016012 

0.63757 

0.05684 

N4017012 

0.00000 

0.00000 

N40 17022 

0.24744 

0.05649 

N4017032 

0.33532 

0.08283 

N4018012 

1.25686 

0.12124 

N40 18022 

1.41914 

0.14693  . 

N40 18032 

0.99172 

0.11152 

N40 18042 

0.79316 

0.10869 

N4019012 

0.45144 

0.09207 

N4020012 

0.75743 

0.06220 

N4020022 

0.74744 

0.06182 

N4020032 

0.00000 

0.00000 

N4020042 

0.00000 

0.00000 

N4020052 

0.75333 

0.07617 

N4021012 

0.79414 

0.06765 

N4022012 

0.28829 

0.03549 

N4024012 

0.57455 

0.14920 

N4025012 

1.12771 

0.12103 

N4026012 

0.00000 

0.00000 

N4026022 

0.42732 

0.00000 

N4026032 

0.00000 

0.00000 

N4027012 

0.67834 

0.07849 

N4028012 

1.56082 

0.10903 

N4029012 

0.42670 

0.12550 

N4030013 

0.37776 

0.05055 

N4031013 

0.43931 

0.04297 

N4032013 

0.46770 

0.03235 

N4032023 

0.33750 

0.03072 

N4033013 

0.54311 

0.04199 

N4034013 

0.43217 

0.05913 

N4035013 

0.48962 

0.05891 

N4035023 

0.53233 

0.04765 

N4035033 

0.34337 

0.05306 

N4036013 

0.80150 

0.07147 

N4037013 

3.50307 

0.00000 

0“  3 

ERIC 


B 

S.E. 

c 

S.E. 

-1.64856 

0.21563 

0.33365 

0.05239 

-0.13604 

0.00000 

0.00000 

0.00000 

-0.04423 

0.10970 

0.44352 

0.02722 

-1.40966 

0.14896 

0.50831 

0.03655 

-0.78651 

0.04697 

0.35748 

0.02081 

-2.05406 

0.26837 

0.49484 

0.05512 

-0.66275 

0.06400 

0.40014 

0.02282 

-0.98284 

0.15213 

0.40990 

0.03660 

0.44668 

0.15774 

0.37533 

0.03338 

1.50077 

0.09544 

0.51612 

0.01490 

1.25531 

0.09690 

0.45069 

0.01892 

0.02681 

0.08091 

0.35528 

0.02445 

0.43991 

0.04365 

0.29307 

0.01686 

1.82967 

0.28756 

0.34420 

0.03575 

0.52351 

0.09799 

0.21851 

0.02659 

1.67353 

0.28317 

0.39704 

0.03904 

2.06877 

0.13179 

0.25864 

0.01377 

0.73835 

0.13570 

0.33332 

0.03104 

1.65079 

0.16694 

0.50029 

0.02241 

-1.07525 

0.17994 

0.38394 

0.04347 

-0.13604 

0.00000 

0.00000 

0.00000 

2.48785 

0.53721 

0.56905 

0.02796 

1.93238 

0.35540 

0.53545 

0.03489 

0.02366 

0.06987 

0.57864 

0.01960 

-0.13923 

0.07391 

0.64604 

0.01901 

0.23069 

0.09425 

0.61889 

0.02030 

1.14505 

0.10423 

0.50150 

0.01971 

2.08817 

0.22929 

0.37678 

0.02702 

-1.10947 

0.14555 

0.41553 

0.03714 

-1.30157 

0.16076 

0.43260 

0.03994 

-0.13604 

0.00000 

0.00000 

0.00000 

-0.13604 

0.00000 

0.00000 

0.00000 

-0.24187 

0.12563 

0.50283 

0.02860 

-0.12558 

0.09179 

0.36789 

0.02683 

0.00296 

0.28100 

0.31156 

0.04313 

2.78002 

0.36401 

0.36424 

0.01848 

1.54349 

0.06302 

0.22499 

0.01247 

-0.13604 

0.00000 

0.00000 

0.00000 

0.20889 

0.21005 

0.57496 

0.02669 

-0.13604 

0.00000 

0.00000 

0.00000 

1.50002 

0.09391 

0.22729 

0.01861 

1.59177 

0.04767 

0.18587 

0.00827 

4.05322 

0.75787 

0.20262 

0.01740 

-6.77791 

0.78455 

0.30681 

0.06678 

-5.26217 

0.44242 

0.30033 

0.06564 

-2.74544 

0.24330 

0.25768 

0.05764 

-1.34363 

0.28629 

0.25354 

0.05200 

-1.32947 

0.18259 

0.27452 

0.04526 

0.50716 

0.19391 

0.37455 

0.03766 

0.11979 

0.18035 

0.39951 

0.03640 

-2.33567 

0.30546 

0.48334 

0.05966 

0.31740 

0.30122 

0.46663 

0.04182 

0.72634 

0.06735 

0.28030 

0.02053 

-0.31270 

0.02007 

0.39230 

0.01571 

801 


NAEPID 

N4037023 

N4037033 

N4038013 

N4038023 

N4038033 

N4038043 

N4039013 

N4040013 

N4042013 


Table  D-31  (continued) 

I RT  Parameters  for  the  Science  Long-Term  Trend  Samples,  Age  9 


A S.E, 

3.35457  0.00000 

3.75914  0.29325 

0.46127  0.09380 

, 0.00000  0.00000 
0.48779  0.05494 

0.40332  0.05095 

0.60614  0.04973 

0.23197  0.03052 

0.49011  0.05656 


B S.E, 


-0.35879 

0.02368 

-0.24694 

0.02539 

1.89190 

0.21792 

-0.13604 

0.00000 

-0.90377 

0.26483 

-0.35973 

0.27889 

-0.46523 

0.12440 

1.09308 

0.26634 

1.24827 

0.10759 

C S.E, 


0.48618  0.01644 

0.39095  0.01599 

0.43910  0.02715 

0.00000  0.00000 
0.47288  0.04632 

0.42061  0.04644 

0.26422  0.03363 

0.24042  0.03499 

0.18109  0.02513 
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IRT  Parameters  for  the 


NAEPID 

A 

S.E. 

N404501 1 

0.85565 

0.05351 

N404601 1 

0.48238 

0.04925 

N4047011 

0.76049 

0.05044 

N4047021 

0.63359 

0.06098 

N400201 1 

0.44659 

0.03463 

N404901 1 

0.76399 

0.04971 

N4049021 

0.00000 

0.00000 

N4049031 

0.00000 

0.00000 

N404801 1 

0.98001 

0.08233 

N4048021 

1.72682 

0.10676 

N4048031 

1.44376 

0.10451 

N4050011 

0.79237 

0.08030 

N4051011 

0.84401 

0.07120 

N4052011 

0.57024 

0.08018 

N4053011 

0.86595 

0.09949 

N405401 1 

1.11274 

0.10370 

N4012011 

1.05210 

0.07084 

N405501 1 

1.19018 

0.09090 

N4056011 

0.75155 

0.12998 

N405701 1 

0.91242 

0.06034 

N4058011 

1.40936 

0.11938 

N405901 1 

1.23781 

0.11235 

N4060011 

0.95940 

0.23629 

N4061011 

1.45895 

0.21427 

N406201 1 

1.42854 

0.13555 

N4063012 

0.56698 

0.12571 

N4063022 

0.33217 

0.03749 

N4063032 

0.87581 

0.08037 

N4063042 

0.61305 

0.11679 

N4064012 

1.01093 

0.11089 

N4064022 

1.25122 

0.10057 

N4064032 

1.05340 

0.11479 

N4064042 

1.24445 

0.10721 

N4064052 

1.12918 

0.09414 

N4065012 

0.77587 

0.07101 

N4066012 

0.49248 

0.05082 

N4067012 

1.08533 

0.09700 

N4068012 

0.80242 

0.06583 

N4068022 

0.45650 

0.08969 

N4068032 

0.67738 

0.05493 

N4068042 

0.72057 

0.05593 

N4068052 

1.56880 

0.15222 

N4068062 

0.38666 

0.04667 

N4069012 

0.70690 

0.07138 

N4070012 

0.42021 

0.06919 

N4071012 

0.82574 

0.11961 

N4072012 

0.80020 

0.08809 

N4073012 

0.44487 

0.05355 

N4073022 

0.89255 

0.17304 

N4080012 

1.43693 

0.10586 

N4076012 

0.73803 

0.09443 

N4075012 

0.00000 

0.00000 

N4077012 

0.76949 

0.08684 

N4078012 

1.36883 

0.14211 

Table  D-32 

Science  Long-Term  Trend  Samples,  Age  13 


B S.E.  C 


-2.31702 

0.13739 

0.25274 

-0.18754 

0.19754 

0.28356 

-1.56840 

0.15228 

0.29493 

0.24217 

0.11615 

0.28737 

-1.83850 

0.24816 

0.28313 

-0.59825 

0.09727 

0.22707 

-0.01344 

0.00000 

0.00000 

-0.01344 

0.00000 

0.00000 

-1.68271 

0.17682 

0.52180 

-0.29858 

0.03993 

0.33931 

0.47822 

0.04162 

0.37175 

0.63614 

0.08311 

0.36227 

0.92283 

0.05519 

0.19584 

0.85935 

0.13560 

0.37345 

1.20749 

0.06828 

0.30316 

1.21835 

0.04767 

0.21978 

0.11518 

0.05553 

0.28190 

0.48914 

0.04989 

0.35287 

1.57379 

0.11259 

0.40943 

0.39984 

0.05042 

0.17925 

1.08156 

0.03773 

0.24713 

1.51164 

0.05485 

0.21477 

2.92186 

0.37582 

0.14820 

2.38895 

0.17170 

0.20091 

2.13437 

0.09698 

0.11224 

1.23073 

0.20393 

0.63128 

-1.02640 

0.30926 

0.47909 

.0.15703 

0.09174 

. 0.41952 

1.45169 

0.14544 

0.47125 

0.26178 

0.09749 

0.59402 

0.26081 

0.05741 

0.43891 

-0.65178 

0.14583 

0.66547 

-0.19117 

0.07711 

0.53874 

-0.26500 

0.08321 

0.49278 

0.74893 

0.06857 

0.23083 

-0.74990 

0.25283 

0.33634 

0.60897 

0.05964 

0.38486 

-1.58961 

0.19118 

0.46833 

1.05124 

0.23627 

0.51134 

-0.79811 

0.15474 

0.36254 

-1.06937 

0.15734 

0.37972 

1.23170 

0.05582 

0.54283 

0.11822 

0.23608 

0.41368 

0.21753 

0.11532 

0.35851 

1.04549 

0.19917 

0.30418 

1.97283 

0.12033 

0.14310 

0.85388 

0.07954 

0.34212 

0.81371 

0.14998 

0.21455 

1.78441 

0.12771 

0.40733 

0.91884 

0.03441 

0.20692 

1.33781 

0.08332 

0.26370 

-0.01344 

0.00000 

0.00000 

1.28984 

0.07242 

0.21377 

1.82767 

0.08538 

0.30380 

803 


S.E. 

0.05922 

0.04856 

0.05606 

0.03444 

0.06071 

0.03502 

0.00000 

0.00000 

0.05700 

0.02046 

0.01667 

0.02502 

0.01849 

0.03399 

0.01896 

0.01388 

0.02237 

0.01874 

0.02075 

0.01980 

0.01264 

0.01054 

0.00821 

0.00728 

0.00615 

0.02801 

0.04298 

0.02785 

0.02676 

0.02311 

0.02068 

0.03147 

0.02447 

0.02701 

0.02331 

0.05996 

0.02013 

0.05768 

0.03882 

0.04490 

0.04753 

0.01132 

0.04088 

0.03352 

0.04493 

0.01384 

0.02395 

0.03776 

0.01704 

0.01292 

0.02176 

0.00000 

0.01990 

0.00966 


Table  D-32  (continued) 

IRT  Parameters  for  the  Science  Long-Term  Trend  Samples,  Age  13 


NAEPID 

N4079012 

N4082012 

N4083013 

N4083023 

N4083033 

N4083043 

N4084013 

N4085013 

N4085023 

N4086013 

N4087013 

N4088013 

N4089013 

N4089023 

N4089033 

N4089043 

N4090013 

N4091013 

N4091023 

N4091033 

N4092013 

N4093013 

N4094013 

N4094023 

N4094033 

N4095013 

N4096013 

N4097013 

N4098013 


A 

S.E. 

B 

S.E. 

c 

S.E. 

0.52343 

0.07264 

1.07908 

0.12737 

0.27277 

0.03386 

1.03478 

0.13878 

1.93198 

0.10689 

0.20136 

0.01125 

1.23038 

0.10140 

0.75319 

0.04690 

0.34605 

0.01656 

0.69316 

0.05783 

-1.22132 

0.19190 

0.44143 

0.05245 

0.68206 

0.07183 

-0.97399 

0.23439 

0.55134 

0.05057 

0.97872 

0.07871 

-1.03037 

0.13047 

0.47376 

0.04153 

0.38108 

0.03850 

-0.29704 

0.22413 

0.31768 

0.04454 

0.78391 

0.05808 

-0.70055 

0.12049 

0.30595 

0.04111 

0.54303 

0.06703 

1.10800 

0.10546 

0.21805 

0.03013 

0.45875 

0.03460 

-1.12937 

0.19781 

0.18206 

0.05332 

0.31063 

0.03907 

0.68616 

0.22743 

0.35566 

0.03556 

0.28896 

0.03712 

0.93120 

0.23154 

0.33292 

0.03391 

1.37529 

0.15554 

0.81009 

0.06066 

0.56976 

0.01544 

0.87551 

0.07293 

-1.75233 

0.19560 

0.50879 

0.06031 

0.92821 

0.08601 

0.52585 

0.07281 

0.40329 

0.02255 

0.78548 

0.13029 

1.39005 

0.10563 

0.47551 

0.02051 

0.47751 

0.03562 

-0.41157 

0.14261 

0.17657 

0.03902 

0.79425 

0.05241 

-1.35261 

0.13641 

0.28649 

0.05089 

0.75910 

0.07188 

0.36214 

0.09339 

0.34899 

0.02856 

0.85745 

0.15687 

1.83667 

0.12733 

0.35671 

0.01667 

0.76617 

0.09905 

1.01019 

0.09027 

0.40541 

0.02393 

0.88858 

0.06127 

-0.07622 

0.07132 

0.24851 

0.02722 

0.00000 

0.00000 

-0.01344 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.01344 

0.00000 

0.00000 

0.00000 

0.00000 

0.00000 

-0.01344 

0.00000 

0.00000 

0.00000 

0.98168 

0.11349 

1.67834 

0.07436 

0.14850 

0.01220 

1.34898 

0.12250 

1.15736 

0.04273 

0.28859 

0.01297 

1.33892 

0.11657 

1.75686 

0.06749 

0.20768 

0.00874 

0.00000 

0.00000 

-0.01344 

0.00000 

0.00000 

0.00000 

O 
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NAEPID 


IRT  Parameters  for  the  Science  Long-Term  Trend  Samples,  Age  17 
A S^E; B S;E; C 


N400201 1 

0.48961 

N4046011 

0.36941 

N4100011 

0.00000 

N4 100021 

0.00000 

N4 100031 

0.24892 

N4 100041 

0.52243 

N4099011 

0.84051 

N408601 1 

0.33388 

N4093011 

0.81322 

N406301 1 

1.15843 

N4063021 

0.30049 

N4063031 

0.89335 

N4063041 

0.48559 

N4101011 

0.70132 

N4101021 

0.32474 

N4101031 

0.35253 

N4066011 

0.37313 

N405001 1 

0.44223 

N4012011 

0.89779 

N4052011 

0.62688 

N4 102011 

0.80127 

N4060011 

0.75794 

N4095011 

0.70233 

N4061011 

1.40936 

N4 103011 

0.00000 

N4062011 

1.19643 

N4081011 

0.65487 

N4064012 

0.97961 

N4064022 

1.01256 

N4064032 

1.13816 

N4064042 

1.17028 

N4064052 

1.01346 

N4104012 

0.22079 

N4068012 

0.77902 

N4068022 

0.28668 

N4068032 

0.76462 

N4068042 

0.64284 

N4068052 

0.55830 

N4068062 

0.37129 

N4 10501 2 

0.34812 

N4106012 

2.22456 

N4 106022 

0.50555 

N4 106032 

1.47739 

N4 106042 

0.46553 

N4069012 

0.53496 

N4074012 

0.42679 

N4074022 

0.00000 

N4074032 

0.57936 

N4074042 

0.62038 

N4072012 

0.64225 

N4070012 

0.34630 

N4107012 

0.92935 

N4077012 

0.74941 

N4073012 

0.29677 

0.04025 

-2.80462 

0.03406 

-1.65536 

0.00000 

-0.05246 

0.00000 

-0.05246 

0.03601 

-4.23989 

0.06168 

-0.86993 

0.06004 

-0.88321 

0.03027 

-2.69910 

0.05208 

-1.39173 

0.12861 

0.24633 

0.03490 

-1.40400 

0.08335 

-0.17972 

0.05506 

-0.40659 

0.07329 

-0.91459 

0.03823 

-1.24174 

0.03945 

-1.52347 

0.03196 

. -2.20500 

0.04925 

-0.25826 

0.06517 

-0.42039 

0.08429 

0.70083 

0.10396 

1.53016 

0.12904 

2.00914 

0.06980 

1.11159 

0.11459 

1.48896 

0.00000 

-0.05246 

0.11440 

1.54744 

0.08758 

1.55297 

0.08256 

-0.50930 

0.07403 

-0.70145 

0.09419 

-1.44138 

0.08626 

-1.07980 

0.07506 

-1.24200 

0.03114 

0.25036 

0.06708 

-2.11823 

0.04220 

1.03988 

0.06229 

-1.19911 

0.05192 

-1.84614 

0.08251 

0.55399 

0.04441 

-0.13503 

0.02922 

-0.84915 

0.10967 

1.13627 

0.04949 

-2.75735 

0.15215 

1.01873 

0.04541 

-2.39292 

0.04910 

-0.60142 

0.04499 

-0.78513 

0.00000 

-0.05246 

0.06571 

-0.20359 

0.05274 

-2.12497 

0.07151 

0.24853 

0.03289 

-0.74863 

0.08270 

0.71901 

0.06744 

0.75671 

0.03616 

0.78352 

0.25554 

0.24478 

0.28311 

0.25768 

0.00000 

0.00000 

0.00000 

0.00000 

0.73728 

0.46308 

0.28723 

0.56137 

0.11818 

0.29998 

0.31234 

0.18504 

0.11897 

0.22096 

0.08882 

0.60036 

0.34079 

0.43034 

0.11360 

0.47337 

0.24253 

0.43123 

0.21985 

0.54313 

0.33313 

0.47067 

0.33548 

0.46857 

0.27163 

0.20399 

0.23159 

0.32492 

0.09336 

0.30911 

0.13071 

0.35856 

0.08655 

0.22590 

0.14467 

0.21074 

0.06951 

0.14791 

0.05064 

0.18525 

0.00000 

0.00000 

0.05618 

0.10057 

0.09689 

0.18004 

0.11104 

0.47105 

0.09797 

0.40396 

0.12954 

0.50314 

0.10004 

0.44047 

0.11888 

0.42634 

0.30688 

0.34747 

0.20115 

0.45913 

0.27208 

0.41353 

0.16460 

0.43715 

0.20103 

0.40048 

0.18465 

0.46707 

0.25009 

0.43904 

0.19391 

0.15372 

0.02537 

0.15519 

0.32733 

0.43501 

0.04812 

0.42845 

0.32611 

0.43134 

0.19111 

0.27961 

0.25679 

0.43222 

0.00000 

0.00000 

0.20025 

0.43983 

0.22815 

0.41582 

0.13791 

0.32865 

0.23552 

0.20844 

0.05909 

0.25772 

0.06816 

0.18867 

0.21737 

0.24971 

ERIC 


S.E. 

0.05804 

0.05745 

0.00000 

0.00000 

0.06551 

0.05027 

0.04496 

0.05437 

0.04809 

0.02252 

0.04610 

0.03321 

0.04958 

0.05078 

0.04557 

0.04938 

0.05572 

0.05137 

0.03533 

0.03581 

0.01826 

0.01721 

0.02139 

0.00961 

0.00000 

0.00949 

0.02221 

0.03587 

0.03663 

0.04808 

0.04070 

0.04533 

0.03659 

0.05877 

0.03338 

0.05001 

0.05618 

0.03855 

0.04098 

0.04173 

0.00800 

0.06644 

0.01415 

0.06484 

0.05223 

0.04798 

0.00000 

0.04578 

0.05960 

0.03981 

0.04850 

0.02116 

0.02360 

0.03662 
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Table  D-33  (continued) 

IRT  Parameters  for  the  Science  Long-Term  Trend  Samples,  Age  17 


NAEPID 

A 

S.E. 

B 

S.E. 

C 

S.E. 

N4073022 

1.12228 

0.15435 

1.24211 

0.06961 

0.43463 

0.01659 

N4071012 

1.09467 

0.10061 

1.11265 

0.04738 

0.19397 

0.01508 

N4108012 

0.36698 

0.05452 

1.25953 

0.19833 

0.24071 

0.04055 

N4109012 

0.99808 

0.07345 

1.02010 

0.04294 

0.10964 

0.01313 

N41 10012 

1.15590 

0.10938 

1.31821 

0.04953 

0.16157 

0.01247 

N4083013 

1.04277 

0.07523 

-0.56072 

0.08889 

0.39743 

0.03349 

N4083023 

0.72177 

0.05706 

-1.87014 

0.18998 

0.42565 

0.05679 

N4083033 

0.81542 

0.06677 

-1.82696 

0.18344 

0.46994 

0.05657 

N4083043 

0.99883 

0.07591 

-1.72258 

0.13779 

0.43647 

0.05176 

N4051013 

1.07687 

0.07790 

0.17801 

0.05752 

0.29439 

0.02324 

N4089013 

0.85016 

0.06708 

-1.06585 

0.14279 

0.45301 

0.04537 

N4089023 

1.31265 

0.11069 

-1.80596 

0.11932 

0.47345 

0.05059 

N4089033 

0.75710 

0.07022 

0.07180 

0.11031 

0.38320 

0.03274 

N4089043 

0.52642 

0.05685 

-0.06095 

0.18789 

0.37462 

0.04400 

N4054013 

0.85491 

0.06394 

0.57100 

0.05480 

0.16494 

0.02102 

N4113013 

0.61534 

0.17737 

3.49267 

0.58316 

0.14062 

0.01258 

N4055013 

0.66249 

0.05133 

-0.50176 

0.12988 

0.27312 

0.04182 

N4111013 

0.51607 

0.04614 

-0.19342 

0.15665 

0.21546 

0.04430 

N41 12013 

0.73900 

0.06302 

0.20167 

0.09177 

0.25667 

0.03143 

N4088013 

0.90586 

0.08028 

-0.01962 

0.09598 

0.40462 

0.03125 

N41 14013 

1.66500 

0.10158 

0.32959 

0.03048 

0.23200 

0.01539 

N4115013 

1.13488 

0.09250 

1.08760 

0.04262 

0.16938 

0.01344 

N4 115023 

0.81636 

0.05719 

-1.06192 

0.12631 

0.31610 

0.04668 

N41 16013 

1.45694 

0.10553 

0.85804 

0.03283 

0.20082 

0.01280 

N41 17013 

1.24097 

0.09496 

0.92225 

0.03803 

0.18924 

0.01388 

N41 18013 

1.70366 

0.10004 

0.35988 

0.02810 

0.20134 

0.01434 

N41 19013 

1.48805 

0.12300 

1.06898 

0.03600 

0.22053 

0.01207 

N4 12001 3 

1.24929 

0.12664 

1.56534 

0.06351 

0.25507 

0.01119 
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Appendix  E 


ESTIMATION  ERROR  VARIANCE 
BY  GENDER  AND  RACE/ETHNICITY 


Table  E-1 

Estimation  Error  Variance  for  the  Main  Mathematics 
Number  Sense,  Properties,  and  Operations  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Variance 


GRADE 4 

Total 

0.88 

Male 

1.33 

Female 

1.10 

White 

1.13 

Black 

7.31 

Hispanic 

4.49 

Asian  American 

17.64 

American  Indian 

6.49 

Other 

53.12 

GRADES 

Total 

1.18 

Male 

1.89 

Female 

1.30 

White 

1.48 

Black 

4.99 

Hispanic 

3.58 

Asian  American 

15.24 

American  Indian 

11.79 

Other 

505.74 

GRADE  12 

Total 

1.27 

Male 

1.73 

Female 

1.53 

White 

1.12 

Black 

5.82 

Hispanic 

3.51 

Asian  American 

27.19 

American  Indian 

111.35 

Other 

141.65 

Student 

Latency 

Samnlins 

of  9 

0.91 

0.09 

0.89 

0.11 

0.85 

0.15 

0.89 

0.11 

0.93 

0.07 

0.89 

0.11 

0.89 

0.11 

0.78 

0.22 

0.76 

0.24 

0.91 

0.09 

0.88 

0.12 

0.85 

0.15 

0.91 

0.09 

0.83 

0.17 

0.76 

0.24 

0.79 

0.21 

0.79 

0.21 

0.94 

0.06 

0.93 

0.07 

0.88 

0.12 

0.91 

0.09 

0.91 

0.09 

0.87 

0.13 

0.77 

0.23 

0.92 

0.08 

0.96 

0.04 

0.89 

0.11 
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Table  E-2 

Estimation  Error  Variance  for  the  Main  Mathematics 
Measurement  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Variance 

Student 

Samnline 

Latency 

ofe 

GRADE 4 

Total 

1.25 

0.84 

0.16 

Male 

1.75 

0.78 

0.22 

Female 

1.61 

0.77 

0.23 

White 

1.63 

0.84 

0.16 

Black 

5.59 

0.75 

0.25 

Hispanic 

6.39 

0.77 

0.23 

Asian  American 

24.93 

0.77 

0.23 

American  Indian 

7.36 

0.75 

0.25 

Other 

128.73 

0.87 

0.13 

GRADES 

Total 

2.06 

0.87 

0.13 

Male 

3.30 

0.86 

0.14 

Female 

2.52 

0.78 

0.22 

White 

2.58 

0.87 

0.13 

Black 

7.81 

0.77 

0.23 

Hispanic 

9.43 

0.74 

0.26 

Asian  American 

29.03 

0.72 

0.28 

American  Indian 

20.90 

0.79 

0.21 

Other 

249.71 

0.78 

0.22 

GRADE  12 

Total 

1.25 

0.84 

0.16 

Male 

1.78 

0.69 

0.31 

Female 

1.78 

0.82 

0.18 

White 

1.39 

0.80 

0.20 

Black 

5.34 

0.72 

0.28 

Hispanic 

5.25 

0.65 

0.35 

Asian  American 

38.41 

0.85 

0.15 

American  Indian 

138.38 

0.94 

0.06 

Other 

101.64 

0.84 

0.16 

o 
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Table  E-3 

Estimation  Error  Variance  for  the  Main  Mathematics 
Geometry  and  Spatial  Sense  Scale 


Proportion  of  Variance  Due  to  . . . 

Total  Student 

Variance  Sampling 


GRADE  4 

Total 

0.77 

0.82 

Male 

1.15 

0.79 

Female 

1.02 

0.74 

White 

0.92 

0.79 

Black 

3.21 

0.66 

Hispanic 

4.26 

0.84 

Asian  American 

20.33 

0.82 

American  Indian 

6.45 

0.73 

Other 

72.49 

0.75 

GRADE  8 

Total 

1.10 

0.91 

Male 

1.73 

0.85 

Female 

1.39 

0.82 

White 

1.32 

0.88 

Black 

4.73 

0.84 

Hispanic 

6.10 

0.85 

Asian  American 

15.43 

0.78 

American  Indian 

13.65 

0.76 

Other 

. 221.87 

0.86 

GRADE  12 

Total 

1.12 

0.88 

Male 

1.44 

0.82 

Female 

1.51 

0.87 

White 

1.38 

0.86 

Black 

5.92 

0.83 

Hispanic 

5.85 

0.84 

Asian  American 

20.32 

0.82 

American  Indian 

67.86 

0.91 

Other 

140.13 

0.91 

Latency 

of0 


0.18 

0.21 

0.26 

0.21 

0.34 

0.16 

0.18 

0.27 

0.25 


0.09 

0.15 

0.18 

0.12 

0.16 

0.15 

0.22 

0.24 

0.14 


0.12 

0.18 

0.13 

0.14 

0.17 

0.16 

0.18 

0.09 

0.09 


Table  E-4 

Estimation  Error  Variance  for  the  Main  Mathematics 
Data  Analysis,  Statistics,  and  Probability  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Variance 

Student 

Samnline 

Latency 
of  9 

GRADE  4 

Total 

1.28 

0.87 

0.13 

Male 

1.61 

0.79 

0.21 

Female 

1.78 

0.82 

0.18 

White 

1.41 

0.84 

0.16 

Black 

10.38 

0.87 

0.13 

Hispanic 

7.58 

0.76 

0.24 

Asian  American 

25.26 

0.77 

0.23 

American  Indian 

7.61 

0.72 

0.28 

Other 

36.14 

0.44 

0.56 

GRADE  8 

Total 

2.31 

0.92 

0.08 

Male 

3.56 

0.89 

0.11 

Female 

2.41 

0.83 

0.17 

White 

3.36 

0.93 

0.07 

Black 

5.95 

0.75 

0.25 

Hispanic 

7.69 

0.76 

0.24 

Asian  American 

27.17 

0.73 

0.27 

American  Indian 

23.20 

0.76 

0.24 

Other 

1034.92 

0.98 

0.02 

GRADE  12 

Total 

0.99 

0.91 

0.09 

Male 

1.45 

0.85 

0.15 

Female 

1.30 

0.84 

0.16 

White 

0.96 

0.85 

0.15 

Black 

6.01 

0.89 

0.11 

Hispanic 

4.11 

0.78 

0.22 

Asian  American 

30.67 

0.91 

0.09 

American  Indian 

70.57 

0.94 

0.06 

Other 

162.78 

0.89 

0.11 
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Table  E-5 

Estimation  Error  Variance  for  the  Main  Mathematics 
Algebra  and  Functions  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Student 

Latency 

Variance 

Samnlins 

of  9 

GRADE  4 

Total 

0.99 

0.83 

0.17 

Male 

1.28 

0.78 

0.22 

Female 

1.38 

0.76 

0.24 

White 

1.09 

0.80 

0.20 

Black 

5.73 

0.75 

0.25 

Hispanic 

4.88 

0.73 

0.27 

Asian  American 

16.94 

0.69 

0.31 

American  Indian 

6.10 

0.71 

0.29 

Other 

49.25 

0.72 

0.28 

GRADES 

Total 

1.19 

0.89 

0.11 

Male 

2.16 

0.91 

0.09 

Female 

1.08 

0.79 

0.21 

White , 

1.61 

0.87 

0.13 

Black 

3.90 

0.78 

0.22 

Hispanic 

4.33 

0.71 

0.29 

Asian  American 

18.11 

0.80 

0.20 

American  Indian 

12.34 

0.78 

0.22 

Other 

441.38 

0.94 

0.06 

GRADE  12 

Total 

1.33 

0.93 

0.07 

Male 

1.70 

0.87 

0.13 

Female 

1.64 

0.88 

0.12 

White 

1.40 

0.92 

0.08 

Black 

8.34 

0.89 

0.11 

Hispanic 

4.33 

0.76 

0.24 

Asian  American 

23.64 

0.88 

0.12 

American  Indian 

63.71 

0.94 

0.06 

Other 

136.92 

0.92 

0.08 

813 
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Table  E-6 

Estimation. Error  Variance  for  the  Main  Mathematics 
Composite  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Student 

Latency 

Variance 

Samnlins 

of  9 

GRADE 4 

Total 

0.78 

0.95 

0.05 

Male 

1.06 

0.95 

0.05 

Female 

0.96 

0.93 

0.07 

White 

0.92 

0.95 

0.05 

Black 

5.07 

0.94 

0.06 

Hispanic 

4.15 

0.93 

0.07 

Asian  American 

16.38 

0.94 

0.06 

American  Indian 

5.14 

0.82 

0.18 

Other 

54.25 

0.83 

0.17 

GRADES 

Total 

1.13 

0.96 

0.04 

Male 

1.88 

0.96 

0.04 

Female 

1.14 

0.94 

0.06 

White 

1.46 

0.96 

0.04 

Black 

3.73 

0.94 

0.06 

Hispanic 

4.20 

0.89 

0.11 

Asian  American 

14.81 

0.93 

0.07 

American  Indian 

8.67 

0.85 

0.15 

Other 

421.34 

0.95 

0.05 

GRADE  12 
Total 

0.99 

0.98 

0.02 

Male 

1.24 

0.95 

0.05 

Female 

1.25 

0.96 

0.04 

White 

0.97 

0.96 

0.04 

Black 

5.14 

0.95 

0.05 

Hispanic 

3.35 

0.92 

0.08 

Asian  American 

23.64 

0.96 

0.04 

American  Indian 

79.94 

0.98 

0.02 

Other 

127.08 

0.94 

0.06 

7'?n 

O 


8.14 


Table  E-7 

Estimation  Error  Variance  for  the  Main  Science 
Earth  Science  Scale 


Proportion  of  Variance  Due  to . . . 


Total 

Student 

Latency 

Variance 

Sampline 

of  9 

GRADE  4 

Total 

0.72 

0.85 

0.15 

Male 

0.98 

0.75 

0.25 

Female 

1.08 

0.80 

0.20 

White 

1.02 

0.85 

0.15 

Black 

5.53 

0.83 

0.17 

Hispanic 

3.69 

0.77 

0.23 

Asian  American 

19.31 

0.83 

0.17 

American  Indian 

20.13 

0.89 

O.II 

Other 

37.63 

0.38 

0.62 

GRADES 

Total 

0.89 

0.91 

0.09 

Male 

1.24 

0.86 

0.14 

Female 

1.46 

0.89 

O.II 

White 

1.30 

0.88 

0.12 

Black 

3.08 

0.78 

0.22 

Hispanic 

2.91 

0.79 

0.21 

Asian  American 

12.67 

0.83 

0.17 

American  Indian 

19.60 

0.91 

0.09 

Other 

28.44 

0.53 

0.47 

GRADE  12 

Total 

0.91 

0.93 

0.07 

Male 

1.82 

0.91 

0.09 

Female 

0.95 

0.87 

0.13 

White 

1.04 

0.92 

0.08 

Black 

3.69 

0.79 

0.21 

Hispanic 

5.87 

0.88 

0.12 

Asian  American 

9.18 

0.84 

0.16 

American  Indian 

19.57 

0.84 

0.16 

Other 

91.69 

0.84 

0.16 

O 
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Table  E-8 

Estimation  Error  Variance  for  the  Main  Science 
Physical  Science  Scale 


Proportion  of  Variance  Due 

Total  Student 

Variance  Sampling 


GRADE  4 

Total 

1.16 

0.82 

Male 

1.62 

0.76 

Female 

1.40 

0.73 

White 

1.53 

0.79 

Black 

4.45 

0.72 

Hispanic 

4.29 

0.69 

Asian  American 

16.42 

0.73 

American  Indian 

13.11 

0.84 

Other 

45.78 

0.45 

GRADE  8 

Total 

0.91 

0.92 

Male 

1.54 

0.88 

Female 

1.26 

0.87 

White 

1.41 

0.91 

Black 

1.59 

0.73 

Hispanic 

3.51 

0.84 

Asian  American 

11.13 

0.77 

American  Indian 

24.17 

0.92 

Other 

40.24 

0.58 

GRADE  12 

Total 

1.03 

0.92 

Male 

1.81 

0.92 

Female 

1.16 

0.88 

White 

1.34 

0.92 

Black 

3.60 

0.84 

Hispanic 

6.16 

0.89 

Asian  American 

9.17 

0.86 

American  Indian 

39.80 

0.89 

Other 

122.19 

0.89 

8JG 


Latency 

of0 


0.18 

0.24 

0.27 

0.21 

0.28 

0.31 

0.27 

0.16 

0.55 


0.08 

0.12 

0.13 

0.09 

0.27 

0.16 

0.23 

0.08 

0.42 


0.08 

0.08 

0.12 

0.08 

0.16 

0.11 

0.14 

0.11 

0.11 


Table  E-9 

Estimation  Error  Variance  for  the  Main  Science 
Life  Science  Scale 


Proportion  of  Variance  Due  to . . . 


Total  Student  Latency 

Variance  Sampling  of  6 


GRADE 4 

Total 

0.86 

0.78 

0.22 

Male 

1.10 

0.77 

0.23 

Female 

1.24 

0.67 

0.33 

White 

1.06 

0.76 

0.24 

Black 

4.07 

0.80 

0.20 

Hispanic 

3.53 

0.72 

0.28 

Asian  American 

17.33 

0.74 

0.26 

American  Indian 

15.78 

0.86 

0.14 

Other 

53.07 

0.59 

0.41 

GRADE  8 

Total 

1.07 

0.86 

0.14 

Male 

1.40 

0.79 

0.21 

Female 

1.77 

0.85 

0.15 

White 

1.40 

0.86 

0.14 

Black 

2.11 

0.68 

0.32 

Hispanic 

4.90 

0.82 

0.18 

Asian  American 

11.07 

0.73 

0.27 

American  Indian 

16.00 

0.85 

0.15 

Other 

66.28 

0.77 

0.23 

GRADE  12 

Total 

0.80 

0.89 

0.11 

Male 

1.35 

0.87 

0.13 

Female 

1.17 

0.85 

0.15 

White 

1.07 

0.89 

0.11 

Black 

1.95 

0.69 

0.31 

Hispanic 

7.38 

0.86 

0.14 

Asian  American 

11.73 

0.82 

0.18 

American  Indian 

27.92 

0.86 

0.14 

Other 

59.92 

0.76 

0.24 
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Table  E-10 

Estimation  Error  Variance  for  the  Main  Science 
Composite  Scale 


Proportion  of  Variance  Due  to . . . 


Total 

Variance 

Student 

Samnlinu 

Latency 
of  6 

GRADE 4 

Total 

0.64 

0.89 

0.11 

Male 

0.87 

0.86 

0.14 

Female 

0.83 

0.84 

0.16 

White 

0.81 

0.87 

0.13 

Black 

3.74 

0.88 

0.12 

Hispanic 

2.80 

0.86 

0.14 

Asian  American 

13.54 

0.91 

0.09 

American  Indian 

14.21 

0.89 

0.11 

Other 

20.64 

0.29 

0.71 

GRADES 

Total 

0.78 

0.94 

0.06 

Male 

1.12 

0.92 

0.08 

Female 

1.27 

0.93 

0.07 

White 

1.10 

0.93 

0.07 

Black 

1.47 

0.84 

0.16 

Hispanic 

3.11 

0.91 

0.09 

Asian  American 

9.11 

0.92 

0.08 

American  Indian 

17.45 

0.92 

0.08 

Other 

29.61 

0.67 

0.33 

GRADE  12 

Total 

0.76 

0.96 

0.04 

Male 

1.44 

0.95 

0.05 

Female 

0.89 

0.94 

0.06 

White 

0.96 

0.95 

0.05 

Black 

2.37 

0.87 

0.13 

Hispanic 

5.50 

0.95 

0.05 

Asian  American 

8.39 

0.94 

0.06 

American  Indian 

23.45 

0.89 

0.11 

Other 

82.38 

0.87 

0.13 

ERIC 
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Table  E-11 

Estimation  Error  Variance  for  the 
Reading  Long-Term  Trend  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Variance 

Student 

Samnline 

Latency 

ofG 

AGE  9 

Total 

1.01 

0.86 

0.14 

Male 

1.88 

0.84 

0.16 

Female 

1.26 

0.79 

0.21 

White 

1.37 

0.85 

0.15 

Black 

5.21 

0.76 

0.24 

Hispanic 

12.36 

0.92 

0.08 

Asian  American 

22.11 

0.87 

0.13 

American  Indian 

200.67 

0.98 

0.02 

Other 

88.41 

0.66 

0.34 

AGE  13 

Total 

0.93 

0.85 

0.15 

Male 

1.51 

0.82 

0.18 

Female 

1.40 

0.78 

0.22 

White 

1.12 

0.79 

0.21 

Black 

5.83 

0.84 

0.16 

Hispanic 

8.81 

0.88 

0.12 

Asian  American 

16.83 

0.82 

0.18 

American  Indian 

69.59 

0.95 

0.05 

Other 

197.80 

0.69 

0.31 

AGE  17 

Total 

1.08 

0.82 

0.18 

Male 

1.92 

0.80 

0.20 

Female 

1.34 

0.67 

0.33 

White 

1.50 

0.80 

0.20 

Black 

5.44 

0.72 

0.28 

Hispanic 

13.50 

0.85 

0.15 

Asian  American 

35.31 

0.85 

0.15 

American  Indian 

123.66 

0.92 

0.08 

Other 

174.14 

0.68 

0.32 
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Table  E-12 

Estimation  Error  Variance  for  the 
Mathematics  Long-Term  Trend  Scale 

Proportion  of  Variance  Due  to . . . 


Total 

Student 

Latency 

Variance 

Samnline 

of  9 

AGE  9 

Total 

0.66 

0.85 

0.15 

Male 

1.48 

0.82 

0.18 

Female 

0.66 

0.68 

0.32 

White 

1.12 

0.88 

0.12 

Black 

2.35 

0.72 

0.28 

Hispanic 

3.13 

0.83 

0.17 

Asian  American 

20.38 

0.82 

0.18 

American  Indian 

16.40 

0.79 

0.21 

Other 

60.95 

0.40 

0.60 

AGE  13 

Total 

0.66 

0.89 

0.11 

Male 

0.99 

0.86 

0.14 

Female 

0.84 

0.86 

0.14 

White 

0.84 

0.89 

0.11 

Black 

1.77 

0.74 

0.26 

Hispanic 

2.94 

0.82 

0.18 

Asian  American 

14.03 

0.92 

0.08 

American  Indian 

73.20 

0.96 

0.04 

Other 

81.97 

0.57 

0.43 

AGE  17 

Total 

1.24 

0.93 

0.07 

Male 

1.68 

0.85 

0.15 

Female 

1.79 

0.93 

0.07 

White 

1.86 

0.93 

0.07 

Black 

3.13 

0.83 

0.17 

Hispanic 

4.66 

0.78 

0.22 

Asian  American 

45.49 

0.95 

0.05 

American  Indian 

98.09 

0.95 

0.05 

Other 

85.23 

0.67 

0.33 

Table  E-13 

Estimation  Error  Variance  for  the 
Science  Long-Term  Trend  Sca.le 


Proportion  of  Variance  Due  to  . . . 


Total 

Student 

Latency 

Variance 

Samnlins 

of  e 

AGE  9 

Total 

1.13 

0.81 

0.19 

Male 

2.22 

0.84 

0.16 

Female 

1.33 

0.60 

0.40 

White 

1.60 

0.79 

0.21 

Black 

7.21 

0.66 

0.34 

Hispanic 

8.58 

0.82 

0.18 

Asian  American 

34.07 

0.83 

0.17 

American  Indian 

47.26 

0.86 

0.14 

Other 

98.41 

0.24 

0.76 

AGE  13 
Total 

1.04 

0.87 

0.13 

Male 

1.23 

0.75 

0.25 

Female 

1.70 

0.82 

0.18 

White 

1.35 

0.83 

0.17 

Black 

4.12 

0.69 

0.31 

Hispanic 

6.18 

0.73 

0.27 

Asian  American 

13.17 

0.75 

0.25 

American  Indian 

47.52 

0.93 

0.07 

Other 

67.10 

0.56 

0.44 

AGE  17 

Total 

1.40 

0.86 

0.14 

Male 

2.92 

0.84 

0.16 

Female 

1.84 

0.80 

0.20 

White 

1.69 

0.85 

0.15 

Black 

6.16 

0.74 

0.26 

Hispanic 

11.81 

0.83 

0.17 

Asian  American 

75.19 

0.94 

0.06 

American  Indian 

282.35 

0.97 

0.03 

Other 

181.29 

0.76 

0.24 
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Table  E-14 

Estimation  Error  Variance  for  the 
Writing  Long-Term  Trend  Scale 


Proportion  of  Variance  Due  to  . . . 


Total 

Variance 

Student 

Samnling 

Latency 

ofe 

AGE  9 

Total 

0.96 

0.87 

0.13 

Male 

1.35 

0.83 

0.17 

Female 

1.58 

0.92 

0.08 

White 

1.03 

0.92 

0.08 

Black 

4.64 

0.83 

0.17 

Hispanic 

4.99 

0.88 

0.12 

Asian  American 

21.40 

0.95 

0.05 

American  Indian 

23.05 

0.94 

0.06 

Other 

141.00 

0.88 

0.12 

AGE  13 

Total 

0.97 

0.87 

0.13 

Male 

1.36 

0.80 

0.20 

Female 

1.11 

0.93 

0.07 

White 

0.90 

0.88 

0.12 

Black 

5.86 

0.88 

0.12 

Hispanic 

3.28 

0.88 

0.12 

Asian  American 

6.00 

0.89 

0.11 

American  Indian 

9.95 

0.91 

0.09 

Other 

60.30 

0.71 

0.29 

AGE  17 

Total 

1.25 

0.93 

0.07 

Male 

1.57 

0.91 

0.09 

Female 

1.56 

0.93 

0.07 

White 

1.50 

0.95 

0.05 

Black 

7.78 

0.94 

0.06 

Hispanic 

4.37 

0.89 

0.11 

Asian  American 

9.66 

0.91 

0.09 

American  Indian 

59.22 

0.96 

0.04 

Other 

76.75 

0.78 

0.22 
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Appendix  F 

SETTING  THE  NAEP  ACHIEVEMENT  LEVELS  FOR 
THE  1996  MATHEMATICS  ASSESSMENT 

Mary  Lyn  Bourque 

National  Assessment  Governing  Board 


Introduction 

Since  1984,  NAEP  has  reported  the  performance  of  students  in  the  nation  and  for  specific 
subpopulations  on  a O-to-500  proficiency  scale.  The  history  and  development  of  the  scale  and  the 
anchoring  procedure  used  to  interpret  specific  points  on  that  scale  is  described  elsewhere  in  this  report. 

The  achievement  levels  reported  in  1996  were  first  developed  in  1992  through  a process 
described  in  more  detail  in  the  following  sections  of  this  chapter.  The  levels  were  based  on  the 
mathematics  assessment  framework  and  item  pools  developed  for  the  1990  and  1992  assessments. 
Although  the  National  Assessment  Governing  Board  (NAGB)  updated  this  framework  in  preparation  for 
the  1996  assessment,  the  differences  in  test  and  item  specifications  were  not  large  enough  to  warrant  a 
new  mathematics  scale.  Therefore,  a decision  was  made  by  the  Board  to  retain  the  same  levels  as 
reported  for  the  1990  and  1992  assessments,  thus  allowing  a third  point  of  comparison  for  the  short-term 
mathematics  trend. 


History  of  the  Achievement  Levels  Development  1990  - 1992 

The  1988  legislation'  created  an  independent  board,  the  National  Assessment  Governing  Board 
(NAGB),  responsible  for  setting  policy  for  the  NAEP  program.  The  Board  has  a statutory  mandate  to 
identify  “appropriate  achievement  goals  for  each  . . . grade  in  each  subject  area  to  be  tested  under  the 
National  Assessment.”  Consistent  with  this  directive,  and  striving  to  achieve  one  of  the  primary  mandates 
of  the  statute  “to  improve  the  form  and  use  of  NAEP  results,”  the  Board  set  performance  standards 
(called  achievement  levels  by  NAGB)  for  the  National  Assessment  in  1990  and  again  in  1992.  The  1994 
legislation  (Public  Law  103-382)  continued  the,  policy  Board  with  slightly  increased  membership,  and  a 
continued  mandate  to  set  student  performance  standards  on  each  age  and  grade  tested. 

The  1990  trial,  initiated  in  December  1989  with  the  dissemination  of  a draft  policy  statement 
(NAGB,  1989)  and  culminating  22  months  later  in  the  publication  of  the  NAGB  report.  The  Levels  of 
Mathematics  Achievement  (Bourque  & Garrison,  1991),  consisted  of  two  phases:  the  main  study  and  a 
replication-validation  study.  Although  there  were  slight  differences  between  the  two  phases,  there  were 
many  common  elements.  Both  phases  used  a modified  (iterative/empirical)  Angoff  (1971)  procedure  for 
arriving  at  the  levels;  both  focused  on  estimating  performance  levels  based  on  a review  of  the  1990 
NAEP  mathematics  item  pool;  and  both  phases  employed  a set  of  policy  definitions  for  Basic,  Proficient, 
and  Advanced  (NAGB,  1990)  as  the  criteria  for  making  the  item  ratings.  However,  the  1990  process  was 
evaluated  by  a number  of  different  groups  (see  Hambleton  & Bourque,  1991)  who  identified  technical 
flaws  in  the  1990  process.  These  evaluations  influenced  NAGB’s  decision  to  set  the  levels  again  in  1992 


' Public  Law  100-297.  (1988).  National  assessment  of  educational  progress  improvement  act  (Article  No.  USC  1221). 
Washington,  DC. 
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and  to  not  use  the  1990  levels  as  benchmarks  for  progress  toward  the  national  goals  during  the  coming 
decade.  However,  it  is  interesting  to  note  that  the  1990  and  1992  processes  produced  remarkably  similar 
results. 


In  September  1991  NAGB  contracted  with  American  College  Testing  (ACT)  to  convene  the 
panels  of  judges  that  would  recommend  the  levels  on  the  1992  NAEP  assessments  in  reading,  writing, 
and  mathematics.  While  the  1992  level-setting  activities  were  not  unlike  those  undertaken  by  NAGB  in 
1990,  there  were  significant  improvements  made  in  the  process  for  1992.  There  was  a concerted  effort  to 
bring  greater  technical  expertise  to  the  process:  The  contractor  selected  by  NAGB  has  a national 
reputation  for  setting  standards  in  a large  number  of  certification  and  licensure  exams;  an  internal  and 
external  advisory  team  monitored  all  the  technical  decisions  made  by  the  contractor  throughout  the 
process;  and  state  assessment  directors  periodically  provided  their  expertise  and  technical  assistance  at 
key  stages  in  the  project. 

Setting  achievement  levels  is  a method  for  setting  standards  on  the  NAEP  assessment  that 
identifies  what  students  should  know  and  be  able  to  do  at  various  points  along  the  proficiency  scale.  The 
initial  policy  definitions  of  the  achievement  levels  were  presented  to  panelists  along  with  an  illustrative 
framework  for  more  indepth  development  and  operationalization  of  the  levels.  Panelists  were  asked  to 
determine  descriptions/definitions  of  the  three  levels  from  the  specific  framework  developed  for  the 
NAEP  assessment  with  respect  to  the  content  and  skills  to  be  assessed.  The  operationalized  definitions 
were  refined  throughout  the  level-setting  process,  as  well  as  validated  with  a supplementary  group  of 
judges  subsequent  to  the  level-setting  meetings.  Panelists  were  also  asked  to  develop  a list  of  illustrative 
tasks  associated  with  each  of  the  levels,  after  which  sample  items  from  the  NAEP  item  pool  were 
identified  to  exemplify  the  full  range  of  performance  of  the  intervals  between  levels.  The  emphasis  in 
operationalizing  the  definitions  and  in  identifying  and  selecting  exemplar  items  and  papers  was  to 
represent  the  full  range  of  performance  from  the  lower  level  to  the  next  higher  level.  The  details  of  the 
implementation  procedures  are  outlined  in  the  remainder  of  this  appendix. 


Preparing  for  the  1992  Mathematics  Level-Setting  Meeting 

It  is  important  for  the  planning  of  any  standard-setting  effort  to  know  how  various  process 
elements  interact  with  each  other.  For  example,  panelists  interact  with  pre-meeting  materials,  the  meeting 
materials  (i.e.,  the  assessment  questions,  rating  forms,  rater  feedback,  and  so  forth),  each  other,  and  the 
project  staff.  All  of  these  elements  combine  to  promote  or  degrade  what  has  been  called  intrajudge 
consistency  and  interjudge  consensus  (Friedman  & Ho,  1990). 

Previous  research  has  conceptualized  the  effects  of  two  major  kinds  of  interaction:  people 
interacting  with  text  (Smith  & Smith,  1988)  and  people  interacting  with  each  other  (Curry,  1987; 
Fitzpatrick,  1989).  To  assess  the  effects  of  textual  and  social  interaction  and  adjust  the  standard  setting 
procedures  accordingly,  a pilot  study  was  conducted  as  the  first  phase  of  the  1992  initiative. 

Reading  was  chosen  as  the  single  content  area  to  be  pilot-tested  since  it  combined  all  of  the 
various  features  found  in  the  other  NAEP  assessments,  including  multiple-choice,  short  constructed- 
response,  and  extended  constructed-response  items.  The  pilot  study  provided  the  opportunity  to 
implement  and  evaluate  all  aspects  of  the  operational  plan — background  materials,  meeting  materials, 
study  design,  meeting  logistics,  staff  function,  and  participant  function. 


The  overall  pilot  effort  was  quite  successful.  The  level-setting  process  worked  well,  and  the  pilot 
allowed  the  contractor  to  make  improvements  in  the  design  before  implementation  activities  began.  For 
example,  schedule  changes  were  made  that  allowed  the  panelists  more  time  to  operationalize  the  policy 
definitions  before  beginning  the  item-rating  task.  Also,  the  feedback  mechanisms  used  to  inform  panelists 
about  interjudge  and  intrajudge  consistency  data  were  improved  for  clarity  and  utility  to  the  entire 
process. 


The  Mathematics  Level-Setting  Panel 

Sixty-nine  panelists  representing  32  jurisdictions  (31  states  and  the  District  of  Columbia)  from 
the  424  nominees  were  invited  to  participate  in  the  level-setting  process.  They  represented  mathematics 
teachers  at  grades  4,  8,  and  12,  nonteacher  educators,  and  members  of  the  noneducator  (general  public) 
community.  The  group  was  balanced  by  gender,  race/ethnicity,  NAEP  regions  of  the  country,  community 
type  (low  SES,  not  low  SES),  district  size,  and  school  type  (public/private).  One  panelist  was  unable  to 
attend  due  to  a family  emergency,  resulting  in  68  participants:  24  at  grade  4 and  22  at  grades  8 and  12. 


Process  for  Developing  the  Achievement  Levels 

The  four-and-one-half  day  session  began  with  a brief  overview  of  NAEP  and  NAGB,  a 
presentation  on  the  policy  definitions  of  the  achievement  levels,  a review  of  the  NAEP  mathematics 
assessment  framework,  and  a discussion  of  factors  that  influence  item  difficulty.  The  purpose  of  the 
presentation  was  to  focus  panelists’  attention  on  the  mathematics  framework  and  to  emphasize  the  fact 
that  panelists’  work  was  directly  related  to  the  NAEP  assessment,  not  to  the  whole  domain  of 
mathematics. 

All  panelists  completed  and  self-scored  an  appropriate  grade-level  form  of  the  NAEP  assessment. 
The  purpose  of  this  exercise  was  to  familiarize  panelists  with  the  test  content  and  scoring  protocols 
before  beginning  to  develop  the  preliminary  operationalized  descriptions  of  the  three  levels. 

Working  in  small  groups  of  five  or  six,  panelists  expanded  and  operationalized  the  policy 
definitions  of  Basic,  Proficient,  and  Advanced  in  terms  of  specific  mathematical  skills,  knowledge,  and 
behaviors  that  were  judged  to  be  appropriate  expectations  for  students  in  each  grade,  and  were  in 
accordance  with  the  current  mathematics  assessment  framework. 

The  policy  definitions  are  as  follows: 

Basic  This  level,  below  proficient,  denotes  partial  mastery  of  the  knowledge  and 

skills  that  are  fundamental  for  proficient  work  at  each  grade — 4,  8,  and  12. 

Proficient  This  central  level  represents  solid  academic  performance  for  each  grade 
tested — 4,  8,  and  12.  Students  reaching  this  level  have  demonstrated 
competency  over  challenging  subject  matter  and  are  well  prepared  for  the 
next  level  of  schooling. 

Advanced  This  higher  level  signifies  superior  performance  beyond  proficient  grade- 
level  mastery  at  grades  4,  8,  and  12. 


825 


741 


The  small  groups  were  allowed  to  brainstorm  about  what  student  performance  should  be,  using 
the  framework  and  their  experience  in  completing  the  NAEP  assessment  as  guides^.  A comprehensive 
listing  of  grade-level  descriptors  was  developed,  and  panelists  were  asked  to  identify  the  five  that  best 
described  what  students  should  be  able  to  do  at  each  of  the  levels.  Those  descriptors  appearing  with  the 
greatest  frequency  were  compiled  into  a discussion  list  for  the  grade-level  groups.  Additions,  deletions, 
and  modifications  were  made  as  a result  of  discussions,  and  the  groups  reached  general  agreement  that 
the  final  list  of  descriptors  represented  what  students  shouldh^  able  to  do  at  each  achievement  level. 

Panelists  next  received  two  hours  of  training  in  the  Angoff  method.  Training  was  customized  to 
reflect  the  unique  item  formats  of  the  particular  subject  area  assessment.  Once  a conceptual  consensus 
was  reached  about  the  characteristics  of  marginally  acceptable  examinees  at  each  of  the  three  levels, 
practice  items  from  the  released  pool  were  rated  by  the  panelists  according  to  the  process  defined  in  the 
contractor’s  plan.  For  multiple-choice  and  short  constructed-response  items,  panelists  were  asked  to  rate 
each  item  for  the  expected  probability  of  a correct  response  for  a group  of  marginally  acceptable 
examinees  at  the  Basic,  Proficient,  and  Advanced  levels.  For  extended  constructed-response  items, 
panelists  were  asked  to  review  20  to  25  student  response  papers  and  select  three  papers,  one  for  each 
achievement  level,  that  typified  marginally  acceptable  examinee  performance  for  that  level. 

Following  training  in  the  Angoff  method,  the  judges  began  the  rating  process,  inspecting  and 
rating  each  item  in  the  pool  for  the  expected  probabilities  of  answering  the  item  correctly  at  each  level. 
Panelists  completed  three  rounds  of  item  ratings.  For  Round  1,  panelists  first  answered  the  items  in  each 
section,  then  reviewed  their  answers  using  scoring  keys  and  protocols.  This  process  helped  ensure  that 
panelists  would  be  thoroughly  familiar  with  each  item,  including  the  foils  and  scoring  rubrics,  before 
rating  the  items.  Panelists  provided  item  ratings/paper  selections  for  all  three  achievement  levels,  one 
item  at  a time,  for  all  the  items  in  a section,  then  proceeded  to  the  next  set  of  items,  for  which  the  process 
was  repeated.  During  Round  1,  panelists  used  their  lists  of  descriptors  and  other  training  materials  for 
guidance  in  the  rating  process. 

Following  Round  1,  item  response  theory  (IRT)  was  used  to  convert  the  rating  results^  for  each 
rater  to  a latent  ability  scale  represented  by  the  Greek  letter  theta  (0).  This  0 scale  was  the  same  scale 
used  to  calibrate  the  NAEP  items  evaluated  by  each  panelist.  In  order  to  provide  meaningful  feedback 
about  item  ratings,  a special  relative  scale  was  constmcted,  which  was  a linear  transformation  of  the 
theta  scale  having  a mean  of  75  and  standard  deviation  of  15.  Before  Round  2 of  the  rating  process, 
panelists  were  given  interjudge  consistency  information  using  this  relative  scale.  This  information 
allowed  panelists  to  see  on  the  scale  where  their  individual  mean  item  ratings  were,  relative  to  the  mean 
for  the  group  and  to  the  means  for  other  panelists.  Reasons  for  extreme  mean  ratings,  including  the 
possibility  that  some  panelists  misinterpreted  the  item  rating  task,  were  discussed  briefly. 

Before  Round  2,  panelists  were  also  given  item  difficulty  data.  This  information  was  presented  as 
the  percentage  of  students  who  answered  each  item  correctly  during  the  actual  NAEP  administration,  for 
items  scored  “correct”  or  “incorrect”  (i.e.,  multiple-choice  and  short  constructed-response  items),  and  as 
the  percentage  of  students  receiving  scores  of  1,  2,  3,  and  4 for  the  extended  constructed-response  items'^. 
Panelists  were  told  that  this  item  difficulty  information  should  be  used  as  a reality  check.  For  items  on 
which  item  ratings  differed  substantially  from  the  item  difficulty  value,  panelists  were  asked  to 


^ The  panelists  also  reviewed  about  half  the  item  pool  (the  half  they  would  not  be  rating  later)  so  the  descriptors  could  be  further 
modified  if  appropriate. 

^ Because  the  IRT  item  parameters  were  not  available  for  the  polytomously  scored  (extended  constructed-response)  items,  these  items 
(five  at  grade  4,  six  each  at  grades  8 and  12)  were  not  included  in  the  following  discussion  of  results. 

^ The  percentages  presented  to  the  raters  summed  to  100  percent,  but  this  excluded  the  percentages — around  80  percent,  in  some 
cases — of  students  who  wrote  responses  that  were  judged  to  be  “off-task,''  those  who  “skipped"  that  question  and  continued  beyond 
that  question,  and  those  who,  apparently,  “never  reached”  that  question. 


reexamine  the  item  to  determine  if  they  had  misinterpreted  the  item  or  misjudged  its  difficulty.  Results  of 
the  data  analysis,  and  panelists’  own  evaluations,  indicated  that  the  item  difficulty  information  was 
perceived  as  very  useful  but  had  little  impact  on  panelists’  ratings. 

For  Round  2,  panelists  reviewed  the  same  set  of  items  they  had  rated  in  Round  1 and,  using  the 
interjudge  consistency  information,  the  item  difficulty  information,  and  the  information  provided  prior  to 
Round  1,  they  either  confirmed  their  initial  item  ratings  or  adjusted  their  ratings  to  reflect  the  additional 
information.  About  one-third  of  Round  1 item  ratings  were  adjusted  during  Round  2. 

Following  Round  2,  panelists’  ratings  were  reanalyzed  and  additional  information  was  presented 
to  panelists  concerning  intrajudge  variability  prior  to  Round  3.  For  each  panelist,  the  intrajudge 
variability  information  consisted  of  those  items  that  they  had  rated  differently  than  items  having  similar 
difficulty,  taking  into  consideration  the  panelist’s  aggregated  item  ratings.  That  is,  the  panelists’ 
aggregated  item  ratings  were  converted  to  the  theta  (0)  scale.  All  items  rated  by  the  panelists  were  then 
analyzed  in  terms  of  the  panelist’s  achievement  level  (0)  in  comparison  to  actual  student  performance  on 
the  items.  The  observed  item  rating  from  each  panelist  was  contrasted  to  an  expected  item  rating.  Those 
items  with  large  differences  between  observed  and  expected  ratings  were  identified.  Panelists  were  given 
this  information  and  asked  to  review  each  of  these  items  and  decide  if  their  Round  2 ratings  still 
accurately  reflected  their  best  judgments  of  the  items.  The  intrajudge  consistency  data  was  to  be  used  to 
flag  items  for  reconsideration  in  the  final  round  of  rating. 

For  Round  3,  panelists  reviewed  the  same  set  of  items  they  rated  in  Rounds  1 and  2 using  both 
the  new  intrajudge  variability  information  and  the  information  made  available  during  Rounds  1 and  2.  In 
addition,  panelists  could  discuss,  within  their  small  groups,  ratings  of  specific  items  about  which  they 
were  unsure.  About  20  percent  of  the  item  ratings  were  adjusted  during  Round  3. 


Process  of  Selecting  Exemplar  Items 

Following  the  standard-setting  meeting,  a series  of  procedures  was  implemented  to  select 
exemplar  items.  First,  expected  and  empirical  p-values  were  computed  for  each  item  in  the  released  item 
pool.  Expected  p-values  were  based  on  predicted  performance  at  the  cut-off  score  for  each  achievement 
level  and  empirical  p-values  were  based  on  the  average  performance  of  all  students  responding  to  the 
item.  Items  that  did  not  have  expected  p-values  h 0.51  for  any  of  the  levels  were  deleted  from  the  item 
pool.  Second,  items  were  compared  to  the  operationalized  descriptions  of  the  levels.  Items  that  did  not 
match  the  content  of  the  descriptions  were  deleted  from  the  item  pool.  Third,  the  remaining  items  were 
classified  as  possible  Basic,  Proficient,  or  Advanced  exemplars  based  on  content  match.  Fourth,  the 
validation  panel  reviewed  the  items  and  recommended  a set  of  items  to  serve  as  exemplars  for  the  levels. 
The  final  set  of  items  was  reviewed  and  approved  by  NAGB  at  their  May  1992  meeting.  These 
procedures  are  described  in  detail  below. 

Using  the  standard-setting  ratings,  expected  p-values  were  computed  for  each  item  at  the  cut 
point  for  each  achievement  level.  The  criteria  described  below  were  applied  to  the  scale-level  results  and 
an  analysis  was  conducted  to  delineate  items  that  could  serve  as  exemplars  for  each  achievement  level 
(Basic,  Proficient,  Advanced). 
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More  specifically,  for  an  item  to  be  chosen  as  a possible  exemplar  for  the  Basic  achievement  level: 


1 . The  expected  p- value  for  students  at  the  cut  point  for  the  Basic  level  of  achievement 
had  to  be  greater  than  0.5 1 ; 

2.  The  content  of  the  item  had  to  match  the  content  of  the  operationalized  description 
of  Basic;  and 

3.  The  empirical  p- value  for  the  item  had  to  be  higher  than  empirical  p- values  for  items 
selected  as  exemplars  for  the  Proficient  level. 


As  an  example: 


Grade  4 Basic  Level  Item  M022801 

Level 

Basic 

Proficient 

Advanced 

Scale  point 

211 

248 

280 

Expected  p-value 

0.70 

0.82 

0.94 

Empirical  p-value 

= 0.52 

For  an  item  to  be  chosen  as  a possible  exemplar  for  the  Proficient  achievement  level: 


1.  The  expected  p- value  for  students  at  the  cut-off  score  for  the  Proficient  level  of 
achievement  had  to  be  greater  than  0.51; 

2.  The  content  of  the  item  had  to  match  the  content  of  the  operationalized  description 
of  Proficient;  and 

3.  The  empirical  p-value  for  the  item  had  to  be  lower  than  empirical  p-values  for  Basic 
exemplar  items,  but  higher  than  student  p-values  for  Advanced  exemplar  items. 


As  an  example: 


Grade  4 Proficient  Level  Item  M022001 


Level 

Basic 

Proficient 

Advanced 

Scale  point 

211 

248 

280 

Expected  p-value 

0.37 

0.58 

0.76 

Empirical  p-value  = 0.35 

744 


For  an  item  to  be  chosen  as  a possible  exemplar  for  the  Advanced  achievement  level: 


1.  The  expected  p-value  for  students  at  the  cut-point  for  the  Advanced  level  of 
achievement  had  to  be  greater  than  0.51; 

2.  The  content  of  the  item  had  to  match  the  content  of  the  operationalized  description 
of  Advanced;  and 

3.  The  empirical  p-value  for  the  item  had  to  be  lower  than  empirical  pwalues  for 
Proficient  exemplar  items. 


As  an  example: 


Grade  4 Advanced  Level  Item  M023101 


Level 

Basic 

ProHcient 

Advanced 

Scale  point 

211 

248 

280 

Expected  p-value 

0.29 

0.43 

0.61 

Empirical  p-value  = 0.22 

The  analysis  procedures  described  above  yielded  31  items  as  possible  grade  4 exemplars,  43  items  as 
possible  grade  8 exemplars,  and  37  items  as  possible  grade  12  exemplars,  as  follows: 


Possible  Exemplar  Items  by  Grade  and  Achievement  Level 


Grade  ’ 

Basic 

ProHcient 

Advanced 

4 

9 

14 

8 

8 

23 

15 

5 

12 

14 

16 

7 

For  grade  4,  the  possible  exemplars  represented  49  percent  of  the  released  item  pool.  For  grades  8 and 
12,  the  possible  exemplars  represented  54  percent  of  the  released  item  pool  for  each  grade. 


Process  for  Validating  the  Levels 

Eighteen  mathematics  educators  participated  in  the  item  selection  and  content  validation  process. 
Ten  of  the  panelists  were  mathematics  teachers  who  had  participated  in  the  original  achievement  levels- 
setting  process  and  who  had  been  identified  as  outstanding  panelists  by  grade  group  facilitators  during 
this  meeting.  The  other  eight  panelists  represented  the  National  Council  of  Teachers  of  Mathematics,  the 
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Mathematical  Sciences  Education  Board,  and  state-level  mathematics  curriculum  supervisors.  To  the 
extent  possible,  the  group  was  balanced  by  race/ethnicity,  gender,  community  type,  and  region  of  the 
country. 

The  two-and-one-half  day  meeting  began  by  briefing  panelists  on  the  purpose  of  the  meeting. 
They  first  reviewed  the  operationalized  descriptions  of  the  achievement  levels  for  consistency  with  the 
NAGB  policy  definitions  of  Basic,  Proficient,  and  Advanced  and  with  the  NAEP  Mathematics 
Objectives.  Next,  they  reviewed  the  operationalized  descriptions  of  the  achievement  levels  for  qualities 
such  as  within-  and  across-grade  consistency,  grade-level  appropriateness,  and  utility  for  increasing  the 
public’s  understanding  of  the  NAEP  mathematics  results.  Finally,  working  first  in  grade  level  (4,  8,  and 
12)  groups  of  six  panelists  each,  then  as  a whole  group,  panelists  revised  the  operationalized  descriptions 
to  provide  more  within-  and  across-grade  consistency  and  to  align  the  language  of  the  description  more 
closely  with  the  language  of  the  NCTM  Standards.  Both  the  original  descriptions  and  the  revised 
descriptions  are  included  later  in  this  appendix. 

On  the  third  day,  panelists  again  split  into  grade-level  groups  of  six  panelists  each  and  reviewed 
the  possible  exemplar  items.  The  task  was  to  select  a set  of  items,  for  each  achievement  level  for  their 
grade,  that  would  best  communicate  to  the  public  the  levels  of  mathematics  ability  and  the  types  of  skills 
needed  to  perform  in  mathematics  at  that  level. 

After  selecting  sets  of  items  for  their  grades,  the  three  grade-level  groups  met  as  a whole  group  to 
review  item  selection.  During  this  process,  cross-grade  items  that  had  been  selected  as  exemplars  by  two 
grade  groups  (three  such  items  were  selected  by  grade  groups  4 and  8)  were  assigned  to  one  grade  by 
whole-group  consensus.  In  addition,  items  were  evaluated  by  the  whole  group  for  overall  quality.  Two 
items  were  rejected  by  the  group  during  this  process  due  to  possible  bias.  This  process  yielded  14  items 
as  recommended  exemplars  for  grade  4,  1 1 items  as  recommended  exemplars  for  grade  8,  and  14  items  as 
recommended  exemplars  for  grade  12. 


Mapping  Panelists’  Ratings  to  the  NAEP  Scales 

The  process  of  mapping  panelists’  ratings  to  the  NAEP  scales  made  significant  use  of  item 
response  theory  (IRT).  IRT  provides  statistically  sophisticated  methods  for  determining  the  expected 
performance  of  examinees  on  particular  test  items  in  terms  of  an  appropriate  measurement  scale.  The 
same  measurement  scale  simultaneously  describes  the  characteristics  of  the  test  items  and  the 
performance  of  the  examinees.  Once  the  item  characteristics  are  set,  it  is  possible  to  precisely  determine 
how  examinees  are  likely  to  perform  on  the  test  items  at  different  points  of  the  measurement  scale. 

The  panelists’  ratings  of  the  NAEP  test  items  were  likewise  linked,  by  definition,  to  the  expected 
performance  of  examinees  at  the  theoretical  achievement  level  cut  points.  It  was  therefore  feasible  to  use 
the  IRT  item  characteristics  to  calculate  the  values  on  the  measurement  scale  corresponding  to  each 
achievement  level.  This  was  done  by  averaging  the  item  ratings  over  panelists  for  each  achievement  level 
and  then  simply  using  the  item  characteristics  to  find  the  corresponding  achievement  level  cut  points  on 
the  IRT  measurement  scale.  This  process  was  repeated  for  each  of  the  NAEP  content  strands  within  each 
grade  (4,  8,  and  12). 

In  the  final  stage  in  the  mapping  process,  the  achievement  level  cut  points  on  the  IRT 
measurement  scale  were  combined  over  content  strands  and  rescaled  to  the  NAEP  score  scale.  Weighted 
averages  of  the  achievement  level  cut  points  were  computed.  The  weighting  constants  accounted  for  the 
measurement  precision  of  the  test  items  evaluated  by  the  panelists,  the  proportion  of  items  belonging  to 


each  NAEP  content  strand,  and  the  linear  NAEP  scale  transformation.  These  weighted  averages  produced 
the  final  cut  points  for  the  Basic,  Proficient,  and  Advanced  achievement  levels  within  each  grade. 


Evaluation  of  the  Mathematics  Levels 

The  1992  mathematics  achievement  levels  used  here  to  report  the  1996  NAEP  data  were 
evaluated  under  a Congressional  mandate  by  the  National  Academy  of  Education  (NAE).  A series  of 
research  studies  were  mounted  by  the  NAE  (National  Academy  of  Education,  1993)  to  look  at  various 
aspects  of  the  validity  of  the  level-setting  process,  and  the  levels  adopted  by  NAGB.  Several  of  these 
studies  focused  specifically  on  the  mathematics  achievement  levels,  and  were  conducted  for  the 
Academy  by  staff  at  the  Learning  Research  and  Development  Center  at  the  University  of  Pittsburgh. 
Based  on  these  studies,  the  Academy’s  1993  policy  report  concluded  that  the  achievement  levels  in 
mathematics  were  flawed  and  should  not  be  continued.  The  more  recent  report  from  the  Academy 
(National  Academy  of  Education,  1997)  concluded  that  the  current  achievement  levels  raised  serious 
concerns  about  their  reliability  and  validity,  were  not  reasonable  (i.e.,  were  set  too  high),  and  in  the  final 
analysis,  should  be  abandoned  by  the  end  of  the  century. 

While  NAGB  did  not  agree  with  the  earlier  policy  report  and  continues  to  disagree  with  the  more 
recent  one,  and  while  the  Board’s  contractor  and  its  technical  advisors  do  not  believe  the  weight  of  the 
evidence  supported  the  NAEP  conclusions,  the  Board  believes  that  standards-based  reporting  is 
responsive  to  the  needs  of  the  users  of  NAEP  data,  is  an  important  aspect  of  the  national  reform 
movement,  and  assists  in  making  NAEP  data  more  useful  and  understandable  to  the  public.  The  Board  is 
committed  to  making  improvements  in  the  process,  and  will  continue  to  support  further  investigation  into 
the  validity  of  the  levels  through  additional  research. 


Figure  F-1 

Final  Description  of  1 992  Mathematics  Achievement  Levels 

GRADE  4 

The  NAEP  content  strands  include:  (1)  number  sense,  properties,  and  operations;  (2)  measurement; 
(3)  geometry;  (4)  data  analysis,  statistics,  and  probability;  (5)  algebra  and  functions.  (Note:  At  the  fourth- 
grade  level,  algebra  and  functions  are  treated  in  informal  and  exploratory  ways,  often  through  the  study  of 
patterns.)  Skills  are  cumulative  across  levels-^rom  Basic  to  Proficient  to  Advanced. 


BASIC.  Fourth-grade  students  performing  at  the  basic  level  should  show  some  evidence  of  understanding 
the  mathematical  concepts  and  procedures  in  the  five  NAEP  content  strands. 

Specifically,  fourth  graders  performing  at  the  basic  level  should  be  able  to  estimate  and  use  basic 
facts  to  perform  simple  computations  with  whole  numbers;  show  some  understanding  of  fractions  and 
decimals;  and  solve  simple  real-world  problems  in  all  NAEP  content  strands.  Students  at  this  level  should 
be  able  to  use — though  not  always  accurately — four-function  calculators,  rulers,  and  geometric  shapes. 
Their  written  responses  are  often  minimal  and  presented  without  supporting  information. 


(continued) 
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Figure  F-1  (continued) 

Final  Description  of  1992  Mathematics  Achievement  Levels 

PROFICIENT.  Fourth-grade  students  performing  at  the  proficient  level  should  consistently  apply 
integrated  procedural  knowledge  and  conceptual  understanding  to  problem  solving  in  the  five  NAEP 
content  strands. 

Specifically,  fourth  graders  performing  at  the  proficient  level  should  be  able  to  use  whole  numbers 
to  estimate,  compute,  and  determine  whether  results  are  reasonable.  They  should  have  a conceptual 
understanding  of  fractions  and  decimals;  be  able  to  solve  real-world  problems  in  all  NAEP  content  strands; 
and  use  four-function  calculators,  rulers,  and  geometric  shapes  appropriately.  Students  performing  at  the 
proficient  level  should  employ  problem-solving  strategies  such  as  identifying  and  using  appropriate 
information.  Their  written  solutions  should  be  organized  and  presented  both  with  supporting  information 
and  explanations  of  how  they  were  achieved. 


ADVANCED.  Fourth-grade  students  performing  at  the  advanced  level  should  apply  integrated  procedural 
knowledge  and  conceptual  understanding  to  complex  and  nonroutine  real-world  problem  solving  in  the  five 
NAEP  content  strands. 

Specifically,  fourth  graders  performing  at  the  advanced  level  should  be  able  to  solve  complex  and 
nonroutine  real-world  problems  in  all  NAEP  content  strands.  They  should  display  mastery  in  the  use  of 
four-function  calculators,  rulers,  and  geometric  shapes.  These  students  are  expected  to  draw  logical 
conclusions  and  justify  answers  and  solution  processes  by  explaining  why,  as  well  as  how,  they  were 
achieved.  They  should  go  beyond  the  obvious  in  their  interpretations  and  be  able  to  communicate  their 
thoughts  clearly  and  concisely. 


GRADES 

NAEP  content  strands:  (1)  number  sense,  properties,  and  operations;  (2)  measurement;  (3) 
geometry;  (4)  data  analysis,  statistics,  and  probability;  (5)  algebra  and  functions.  Skills  are  cumulative 
across  all  levels — from  Basic  to  Proficient  to  Advanced. 


BASIC.  Eighth-grade  students  performing  at  the  basic  level  should  exhibit  evidence  of  conceptual  and 
procedural  understanding  in  the  five  NAEP  content  strands.  This  level  of  performance  signifies  an 
understanding  of  arithmetic  operations — including  estimation — on  whole  numbers,  decimals,  fractions,  and 
percents. 

Eighth  graders  performing  at  the  basic  level  should  complete  problems  correctly  with  the  help  of 
structural  prompts  such  as  diagrams,  charts,  and  graphs.  They  should  be  able  to  solve  problems  in  all  NAEP 
content  strands  through  the  appropriate  selection  and  use  of  strategies  and  technological  tools — including 
calculators,  computers,  and  geometric  shapes.  Students  at  this  level  also  should  be  able  to  use  fundamental 
algebraic  and  informal  geometric  concepts  in  problem  solving. 

As  they  approach  the  proficient  level,  students  at  the  basic  level  should  be  able  to  determine  which 
of  available  data  are  necessary  and  sufficient  for  correct  solutions  and  use  them  in  problem  solving. 
However,  these  eighth  graders  show  limited  skill  in  communicating  mathematically. 
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Figure  F-1  (continued) 

Final  Description  of  1992  Mathematics  Achievement  Levels 

PROFICIENT.  Eighth-grade  students  performing  at  the  proficient  level  should  apply  mathematical 
concepts  and  procedures  consistently  to  complex  problems  in  the  five  NAEP  content  strands. 

They  should  be  able  to  conjecture,  defend  their  ideas,  and  give  supporting  examples.  They  should 
understand  the  connections  between  fractions,  percents,  decimals,  and  other  mathematical  topics  such  as 
algebra  and  functions.  Students  at  this  level  are  expected  to  have  a thorough  understanding  of  basic-level 
arithmetic  operations — an  understanding  sufficient  for  problem  solving  in  practical  situations. 

Quantity  and  spatial  relationships  in  problem  solving  and  reasoning  should  be  familiar  to  them,  and 
they  should  be  able  to  convey  underlying  reasoning  skills  beyond  the  level  of  arithmetic.  They  should  be 
able  to  compare  and  contrast  mathematical  ideas  and  generate  their  own  examples.  These  students  should 
make  inferences  from  data  and  graphs;  apply  properties  of  informal  geometry;  and  accurately  use  the  tools 
of  technology.  Students  at  this  level  should  understand  the  process  of  gathering  and  organizing  data  and  be 
able  to  calculate,  evaluate,  and  communicate  results  within  the  domain  of  statistics  and  probability. 


ADVANCED.  Eighth-grade  students  at  the  advanced  level  should  be  able  to  reach  beyond  the  recognition, 
identification,  and  application  of  mathematical  rules  in  order  to  generalize  and  synthesize  concepts  and 
principles  in  the  five  NAEP  content  strands. 

They  should  be  able  to  probe  examples  and  counter  examples  in  order  to  shape  generalizations  from 
which  they  can  develop  models.  Eighth  graders  performing  at  the  advanced  level  should  use  number  sense 
and  geometric  awareness  to  consider  the  reasonableness  of  an  answer.  They  are  expected  to  use  abstract 
thinking  to  create  unique  problem-solving  techniques  and  explain  the  reasoning  processes  underlying  their 

conclusions. 


GRADE  12 

NAEP  content  strands:  (1)  number  sense,  properties,  and  operations;  (2)  measurement;  {3) 
geometry;  (4)  data  analysis,  statistics,  and  probability;  (5)  algebra  and  functions.  Skills  are  cumulative 

across  levels — from  Basic  to  Proficient  to  Advanced. 

BASIC.  Twelfth-grade  students  at  the  basic  level  should  demonstrate  procedural  and  conceptual 
knowledge  in  solving  problems  in  the  five  NAEP  content  strands. 

They  should  be  able  to  use  estimation  to  verify  solutions  and  determine  the  reasonableness  of 
results  as  applied  to  real-world  problems.  They  are  expected  to  use  algebraic  and  geometric  reasoning 
strategies  to  solve  problems.  Twelfth  graders  performing  at  the  basic  level  should  recognize  relationships 
presented  in  verbal,  algebraic,  tabular,  and  graphical  forms;  and  demonstrate  knowledge  of  geometnc 
relationships  and  corresponding  measurement  skills. 

Twelfth  graders  at  the  basic  level  should  be  able  to  apply  statistical  reasoning  in  the  organization 
and  display  of  data  and  in  reading  tables  and  graphs.  They  also  should  be  able  to  generalize  from  patterns 
and  examples  in  the  areas  of  algebra,  geometry,  and  statistics.  At  this  level,  they  should  use  correct 
mathematical  language  and  symbols  to  communicate  mathematical  relationships  and  reasoning  processes; 
and  use  calculators  appropriately  to  solve  problems. 
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Figure  F-1  (continued) 

Final  Description  of  1992  Mathematics  Achievement  Levels 

PROFICIENT.  Twelfth-grade  students  at  the  proficient  level  should  consistently  integrate  mathematical 
concepts  and  procedures  to  the  solutions  of  more  complex  problems  in  the  five  NAEP  content  strands. 

Twelfth  graders  performing  at  the  proficient  level  should  demonstrate  an  understanding  of 
algebraic,  statistical,  and  geometric  and  spatial  reasoning.  They  should  be  able  to  perform  algebraic 
operations  involving  polynomials;  justify  geometric  relationships;  and  judge  and  defend  the  reasonableness 
of  answers  as  applied  to  real-world  situations.  These  students  should  be  able  to  analyze  and  interpret  data  in 
tabular  and  graphical  form;  understand  and  use  elements  of  the  function  concept  in  symbolic,  graphical,  and 
tabular  form;  and  make  conjectures,  defend  ideas,  and  give  supporting  examples. 


ADVANCED.  Twelfth-grade  students  at  the  advanced  level  should  consistently  demonstrate  the 
integration  of  procedural  and  conceptual  knowledge  and  the  synthesis  of  ideas  in  the  five  NAEP  content 
strands. 


They  should  understand  the  function  concept;  and  be  able  to  compare  and  apply  the  numeric, 
algebraic,  and  graphical  properties  of  functions.  They  should  apply  their  knowledge  of  algebra,  geometry,' 
and  statistics  to  solve  problems  in  more  advanced  areas  of  continuous  and  discrete  mathematics. 

Twelfth  graders  performing  at  the  advanced  level  should  be  able  to  formulate  generalizations  and 
create  models  through  probing  examples  and  counterexamples.  They  are  expected  to  communicate  their 
mathematical  reasoning  through  the  clear,  concise,  and  correct  use  of  mathematical  symbolism  and  logical 
thinking. 


Figure  F-2 

Draft  Descriptions  of  the  Achievement  Levels 
Prepared  by  the  Original  Level-Setting  Panel 

Fourth-Grade  Draft  Descriptions 

BASIC.  The  Basic  level  signifies  some  evidence  of  conceptual  and  procedural  understanding  in  the  five 
NAEP  content  strands  of  number  sense,  properties,  and  operations;  measurement;  geometry  and  spatial 
sense;  data  analysis,  statistics,  and  probability;  and  algebra  and  functions.  Understanding  simple  facts  and 
single-step  operations  are  included  at  this  level,  as  is  the  ability  to  perform  simple  computations  with  whole 
numbers.  This  level  shows  a partial  mastery  of  estimation,  basic  fractions,  and  decimals  relating  to  money  or 
the  number  line;  it  shows  an  ability  to  solve  simple  real-world  problems  involving  measurement, 
probability,  statistics,  and  geometry.  At  this  level,  there  is  a partial  mastery  of  tools  such  as  four-function 
calculators  and  manipulatives  (geometric  shapes  and  rulers).  Written  responses  are  often  minimal,  perhaps 
with  a partial  response  and  lack  of  supportive  information. 


834 


(continued) 


Figure  F-2  (continued) 

Draft  Descriptions  of  the  Achievement  Levels 
Prepared  by  the  Original  Level-setting  Panel 

PROFICIENT.  The  Proficient  level  signifies  consistent  demonstration  of  the  integration  of  procedural 
knowledge  and  conceptual  understanding  as  applied  to  problem  solving  in  the  five  NAEP  content  strands  of 
number  sense,  properties,  and  operations;  measurement;  geometry  and  spatial  sense,  data  analysis, 
statistics,  and  probability;  and  algebra  and  functions.  The  Proficient  level  indicates  an  ability  to  perform 
computation  and  estimation  with  whole  numbers,  to  identify  fractions,  and  to  work  with  decimals  involving 
money  or  the  number  line.  Solving  real-world  problems  involving  measurement,  probability,  statistics,  and 
geometry  is  an  important  part  of  this  level.  This  level  signifies  the  ability  to  use,  as  tools,  four-function 
calculators,  rulers,  and  manipulatives  (geometric  shapes).  It  includes  the  ability  to  identify  and  use 
pertinent/appropriate  information  in  problem  settings.  The  ability  to  make  connections  between  and  among 
skills  and  concepts  emerges  at  this  level.  Clear  and  organized  written  presentations,  with  supportive 
information,  are  typical.  And,  there  is  an  ability  to  explain  how  the  solution  was  achieved. 


ADVANCED.  The  Advanced  level  signifies  the  integration  of  procedural  knowledge  and  conceptual 
understanding  as  applied  to  problem  solving  in  the  five  NAEP  content  strands  of  number  sense,  properties, 
and  operations;  measurement;  geometry  and  spatial  sense;  data  analysis,  statistics,  and  probability,  and 
algebra  and  functions.  This  is  evidenced  by  divergent  and  elaborate  written  responses.  The  Advanced  level 
indicates  an  ability  to  solve  multistep  and  nonroutine  real-world  problems  involving  measurement, 
probability,  statistics,  and  geometry,  and  an  ability  to  perform  complex  tasks  involving  multiple  steps  and 
variables.  Tools  are  mastered,  including  four-function  calculators,  rulers,  and  manipulatives  (geometric 
shapes).  This  level  signifies  the  ability  to  apply  facts  and  procedures  by  explaining  why  as  well  as  how. 
Interpretations  extend  beyond  obvious  connections  and  thoughts  are  communicated  clearly  and  concisely. 
At  this  level,  logical  conclusions  can  be  drawn  and  complete  justifications  can  be  provided  for  answers 
and/or  solution  processes. 


Eighth-Grade  Draft  Descriptions 

BASIC.  Basic  students  should  begin  to  describe  objects,  to  process  accurately  and  elaborate  relationships, 
to  compare  and  contrast,  to  find  patterns,  to  reason  from  graphs,  and  to  understand  spatial  reasoning.  This 
level  of  partial  mastery  signifies  an  understanding  of  arithmetic  operations  on  whole  numbers,  decimals, 
fractions,  and  percents,  including  estimation.  Problems  that  are  already  set  up  are  generally  solved  correctly, 
as  are  one-step  problems.  However,  problems  involving  the  use  of  available  data,  and  determinations  of 
what  is  necessary  and  sufficient  to  solve  the  problem,  are  generally  quite  difficult.  Students  should  select 
appropriate  problem-solving  tools,  including  calculators,  computers,  and  manipulatives  (geometric  shapes) 
to  solve  problems  from  the  five  content  strands.  Students  should  also  be  able  to  use  elementary  algebraic 
concepts  and  elementary  geometric  concepts  to  solve  problems.  This  level  indicates  familiarity  with  the 
general  characteristics  of  measurement.  Students  at  this  level  may  demonstrate  limited  ability  to 
communicate  mathematical  ideas. 
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Figure  F-2  (continued) 

Draft  Descriptions  of  the  Achievement  Levels 

Prepared  by  the  Original  Level-setting  Panel 

PROFICIENT.  Proficient  students  apply  mathematical  concepts  consistently  to  more  complex  problems. 
They  should  make  conjectures,  defend  their  ideas,  and  give  supporting  examples.  They  have  developed  the 
ability  to  relate  the  connections  between  fractions,  percents,  and  decimals,  as  well  as  other  mathematical 
topics.  The  Proficient  level  denotes  a thorough  understanding  of  the  arithmetic  operations  listed  at  the  Basic 
level.  This  understanding  is  sufficient  to  permit  applications  to  problem  solving  in  practical  situations. 
Quantity  and  spatial  relationships  are  familiar  situations  for  problem  solving  and  reasoning,  and  this  level 
signifies  an  ability  to  convey  the  underlying  reasoning  skills  beyond  the  level  of  arithmetic.  Ability  to 
conipare  and  contrast  mathematical  ideas  and  generating  examples  is  within  the  Proficient  domain. 
Proficient  students  can  make  inferences  from  data  and  graphs;  they  understand  the  process  of  gathering  and 
organizing  data,  calculating  and  evaluating  within  the  domain  of  statistics  and  probability,  and 
communicating  the  results.  The  Proficient  level  includes  the  ability  to  apply  the  properties  of  elementary 
geometry.  Students  at  this  level  should  accurately  use  the  appropriate  tools  of  technology. 

ADVANCED.  The  Advanced  level  is  characterized  by  the  ability  to  go  beyond  recognition,  identification, 
^d  application  of  mathematical  rules  in  order  to  generalize  and  synthesize  concepts  and  principles! 
Generalization  often  takes  shape  through  probing  examples  and  counterexamples  and  can  be  focused  toward 
creating  models.  Mathematical  concepts  and  relationships  are  frequently  communicated  with  mathematical 
language,  using  symbolic  representations  where  appropriate.  Students  at  the  Advanced  level  consider  the 
reasonableness  of  an  answer,  with  both  number  sense  and  geometric  awareness.  Their  abstract  thinking 
ability  allows  them  to  create  unique  problem-solving  techniques  and  explain  the  reasoning  processes  they 
followed  in  reaching  a conclusion.  These  students  can  probe  through  examples  and  counterexamples  that 
allow  generalization  and  description  of  assumptions  with  models  and  elegant  mathematical  language. 

Twelfth-Grade  Draft  Descriptions 

BASIC.  This  level  represents  understanding  of  fundamental  algebraic  operations  with  real  numbers, 
including  the  ability  to  solve  two-step  computational  problems.  It  also  signifies  an  understanding  of 
element^  geometrical  concepts  such  as  area,  perimeter,  and  volume,  and  the  ability  to  make  measurements 
of  length,  weight,  capacity,  and  time.  Also  included  at  the  Basic  level  is  the  ability  to  comprehend  data  in 
both  tabular  and  graphical  form  and  to  translate  between  verbal,  algebraic,  and  graphical  forms  of  linear 
expression.  Students  at  this  level  should  be  able  to  use  a calculator  appropriately. 

PROFICIENT.  This  level  represents  mastery  of  fundamental  algebraic  operations  and  concepts  with  real 
numbers,  ^d  an  understanding  of  complex  numbers.  It  also  represents  understanding  of  polynomials  and 
their  graphs  up  to  the  second  degree,  including  conic  sections.  The  elements  of  plane,  solid,  and  coordinate 
geometry  should  be  understood  at  the  Proficient  level.  The  Proficient  level  includes  the  ability  to  apply 
concepts  and  formulas  to  problem  solving.  Students  at  this  level  should  demonstrate  critical  thinking  skills 
The  Proficient  level  also  represents  the  ability  to  Judge  the  reasonableness  of  answers  and  the  ability  to 
analyze  and  interpret  data  in  both  tabular  and  graphical  form.  Basic  algebraic  concepts,  measurement  and 
constructive  geometry  concepts  are  mastered  at  this  level. 

ADVANCED.  The  Advanced  level  represents  mastery  of  trigonometric,  exponential,  logarithmic,  and 
composite  functions,  zeros  and  inverses  of  functions,  polynomials  of  the  third  degree  and  higher,  rational 
functions,  and  graphs  of  all  of  these.  In  addition,  the  Advanced  level  represents  mastery  of  topics  iii  discrete 
mathematics  including  matrices  and  determinants,  sequences  and  series,  and  probability  and  statistics,  as 
well  as  topics  in  analytic  geometry.  The  Advanced  level  also  signifies  the  ability  to  successfully  apply  these 
concepts  to  a variety  of  problem-solving  situations. 
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Figure  F-3 

Revised  Draft  Descriptions  of  the  Achievement  Levels 
Recommended  by  the  Follow-Up  Validation  Panel 


Revised  Fourth-Grade  Draft  Descriptions 

BASIC.  Basic  students  exhibit  some  evidence  of  conceptual  and  procedure  understanding  in  the 
five  NAEP  content  strands.  At  the  fourth  grade  level,  algebra  and  functions  are  treated  in  informal  and 
exploratory  ways  often  through  the  study  of  patterns.  Basic  students  estimate  and  use  basic  facts  to  perfoim 
simple  computations  with  whole  numbers.  These  students  show  some  understanding  of  fractions  and 
decimals.  They  solve  simple  real  world  problems  in  all  areas.  These  students  use,  although  not  always 
accurately,  four-function  calculators,  rulers,  and  geometric  shapes.  Written  responses  are  often  minimal  and 
lack  supporting  information. 


PROFICIENT.  Proficient  students  consistently  integrate  procedural  knowledge  and  conceptual 
understanding  as  applied  to  problem  solving  in  the  five  NAEP  content  strands.  Using  whole  numbers  they 
estimate,  compute,  and  determine  whether  their  results  are  reasonable.  They  have  a conceptual 
understanding  of  fractions  and  decimals.  Solving  real  world  problems  in  all  areas  is  important  at  this  level. 
Proficient  students  appropriately  use  four-function  calculators,  rulers  and  geometric  shapes.  These  students 
use  problem  solving  strategies  such  as  identifying  and  using  appropriate  information.  [Problem-solving 
strategies  include  identification  and  use  of  appropriate  information.]  They  present  organized  written 
solutions  with  supporting  information  and  explain  how  they  were  achieved. 


ADVANCED.  Advanced  students  integrate  procedural  knowledge  and  conceptual  understanding  as 
applied  to  problem  solving  in  the  five  NAEP  content  strands.  They  solve  complex  and  nonroutine  real-world 
problems  in  all  areas.  They  have  mastered  the  use  of  tools  such  as  four-function  calculators,  rulers  and 
geometric  shapes.  Advanced  students  draw  logical  conclusions  and  justify  answers  and  solution  processes 
by  explaining  the  “why”  as  well  as  the  “how.”  Interpretations  extend  beyond  obvious  connections  and 
thoughts  are  communicated  clearly  and  concisely. 


Revised  Eighth-Grade  Draft  Descriptions 

BASIC.  Basic  students  exhibit  evidence  of  conceptual  and  procedural  understanding.  These 
students  compare  and  contrast,  find  patterns,  reason  from  graphs,  and  understand  spatial  reasoning.  This 
level  of  performance  signifies  an  understanding  of  arithmetic  operations,  including  estimation,  on  whole 
numbers,  decimals,  fractions,  and  percents.  Students  complete  problems  correctly  with  the  help  of  structural 
prompts  such  as  diagrams,  charts,  and  graphs.  As  students  approach  the  proficient  level,  they  will  solve 
problems  involving  the  use  of  available  data,  and  determine  what  is  necessary  and  sufficient  for  a correct 
solution.  Students  use  problem  solving  strategies  and  select  appropriate  tools,  including  calculators, 
computers,  and  manipulatives  (geometric  shapes)  to  solve  problems  from  the  five  content  strands.  Students 
use  fundamental  algebraic  and  informal  geometric  concepts  to  solve  problems.  Students  at  this  level 
demonstrate  limited  skills  in  communicating  mathematically. 

(continued) 


837 


753 


Figure  F-3  (continued) 

Revised  Draft  Descriptions  of  the  Achievement  Levels 
Recommended  by  the  Follow-Up  Validation  Panel 

PROFICIENT.  Proficient  students  apply  mathematical  concepts  and  procedures  consistently  to 
complex  problems.  They  make  conjectures,  defend  their  ideas,  and  give  supporting  examples.  They  have 
developed  the  ability  to  relate  the  connections  between  fractions,  percents,  and  decimals,  as  well  as  other 
mathematical  topics,  such  as  algebra  and  functions.  The  proficient  level  denotes  a thorough  understanding 
of  the  arithmetic  operations  listed  at  the  basic  level.  This  understanding  is  sufficient  to  permit  applications 
to  problem  solving  in  practical  situations.  Quantity  and  spatial  relationships  are  familiar  situations  for 
problem  solving  and  reasoning,  and  students  at  this  level  convey  the  underlying  reasoning  skills  beyond  the 
level  of  arithmetic.  Proficient  students  compare  and  contrast  mathematical  ideas  and  generate  their  own 
examples.  These  students  make  inferences  from  data  and  graphs;  they  understand  the  process  of  gathering 
and  organizing  data,  calculating,  evaluating,  and  communicating  the  results  within  the  domain  of  statistics 
and  probability.  Proficient  students  apply  the  properties  of  informal  geometry,  and  accurately  use  the 
appropriate  tools  of  technology. 


ADVANCED.  Advanced  students  go  beyond  recognition,  identification,  and  application  of 
mathematical  rules  in  order  to  generalize  and  synthesize  concepts  and  principles.  Generalization  often  takes 
shape  through  probing  examples  and  counter  examples  and  can  be  used  to  create  models.  Mathematical 
concepts  and  relationships  are  frequently  communicated  with  mathematical  language,  using  symbolic 
representations  where  appropriate.  Students  at  the  advanced  level  consider  the  reasonableness  of  an  answer, 
with  both  number  sense  and  geometric  awareness.  Their  abstract  thinking  allows  them  to  create  unique 
problem  solving  techniques  and  explain  the  reasoning  processes  they  followed  in  reaching  a conclusion. 
These  students  probe  examples  and  counter  examples  that  allow  generalization  and  description  of 
assumptions  with  models  and  elegant  mathematical  language. 


Revised  Twelfth-Grade  Draft  Descriptions 


BASIC.  Basic  students  demonstrate  procedural  and  conceptual  knowledge  in  solving  problems  in 
the  five  NAEP  content  strands.  They  use  estimation  to  verify  solutions  and  determine  the  reasonableness  of 
the  results  to  real  world  problems.  Algebraic  and  geometric  reasoning  strategies  are  used  to  solve  problems. 
These  students  recognize  relationships  in  verbal,  algebraic,  tabular,  and  graphical  forms.  Basic  students 
demonstrate  knowledge  of  geometric  relationships  as  well  as  corresponding  measurement  skills.  Statistical 
reasoning  is  applied  to  the  organization  and  display  of  data  and  to  reading  tables  and  graphs.  These  students 
generalize  from  patterns  and  examples  in  the  areas  of  algebra,  geometry,  and  statistics.  They  communicate 
mathematical  relationships  and  reasoning  processes  with  correct  mathematical  language  and  symbolic 
representations.  Calculators  are  used  appropriately  to  solve  problems. 

PROFICIENT.  Proficient  students  integrate  mathematical  concepts  and  procedures  consistently  to 
more  complex  problems  in  the  five  NAEP  content  strands.  They  demonstrate  an  understanding  of  algebraic 
reasoning,  geometric  and  spatial  reasoning,  and  statistical  reasoning  as  applied  to  other  areas  of 
mathematics.  They  perform  algebraic  operations  involving  polynomials,  justify  geometric  relationships,  and 
judge  and  defend  the  reasonableness  of  answers  in  real  world  situations.  These  students  analyze  and 
interpret  data  in  tabular  and  graphical  form.  Proficient  students  understand  and  use  elements  of  the  function 

concept  in  symbolic,  graphical  and  tabular  form.  They  make  conjectures,  defend  their  ideas,  and  give 
supporting  examples. 
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Figure  F-3  (continued) 

Revised  Draft  Descriptions  of  the  Achievement  Levels 
Recommended  by  the  Follow-Up  Validation  Panel 

ADVANCED.  Advanced  students  consistently  demonstrate  the  integration  of  procedural  and 
conceptual  knowledge,  as  well  as  the  synthesis  of  ideas,  in  the  five  NAEP  content  strands.  Advanced 
students  understand  the  function  concept,  and  they  compare  and  apply  the  numeric,  algebraic,  and  graphical 
properties  of  functions.  They  apply  and  connect  their  knowledge  of  algebra,  geometry,  and  statistics  to  solve 
problems  in  more  advanced  areas  of  continuous  and  discrete  mathematics.  Advanced  smdents  formulate 
generalizations  using  examples  and  counter  examples  to  create  models.  In  communicating  their 
mathematical  reasoning,  these  students  demonstrate  clear,  concise,  and  correct  use  of  mathematical 
symbolism  and  logical  thinking. 


Figure  F-4 

Meeting  Participants,  NAEP  Mathematics  Achievement  Level-Setting 
Original  Meeting,  St.  Louis,  Missouri,  March  20-24,  1992 
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Figure  F-4  (continued) 

Meeting  Participants,  NAEP  Mathematics  Achievement  Level-Setting 
Original  Meeting,  St.  Louis,  Missouri,  March  20-24,  1992 
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Figure  F-5 

Meeting  Participants,  NAEP  Mathematics  Achievement  Level-Setting 
Follow-up  Validation  Meeting,  Nantucket,  Massachusetts,  July  17-19,  1992 
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REPORT  ON  DEVELOPING  ACHIEVEMENT  LEVEL  DESCRIPTIONS 
FOR  THE  1996  NAEP  SCIENCE  ASSESSMENT 

Mary  Lyn  Bourque 

National  Assessment  Governing  Board 


Background 

Principle  4 of  the  Board’s  policy  states  that  the  Board  shall  exercise  its  policy  judgment  in  setting 
the  levels.  That  is,  the  Board’s  statutory  authority  requires  that  it  be  the  final  judge  in  developing  the 
achievement  levels.  In  so  doing,  the  Board  uses  a national  consensus  process,  seeking  advice  from  a broad 
audience.  The  preliminary  descriptions,  for  example,  are  widely  circulated  as  part  of  the  assessment 
framework  documents  before  adoption  by  the  Board.  Similarly,  in  designing  the  level  setting  process  the 
Board  solicits  broad  input  from  relevant  professional  organizations,  the  technical  community,  and  others. 
And  after  the  proposed  levels  are  recommended,  they  are  generally  widely  circulated  for  comment  and 
reflection  by  stakeholder  groups,  policymakers,  content  specialists,  and  others. 

The  Boaid’s  current  decision  in  science  considered  several  evaluative  features  of  the  student 
performance  standards,  including  the  reasonableness  of  the  levels,  whether  they  are  valid,  and  whether  they 
are  likely  to  be  useful  to  the  public  in  interpreting  the  NAEP  data.  These  evaluation  criteria  are  part  of  the 
authorizing  legislation,  and  are  among  the  criteria  against  which  the  National  Academy  of  Sciences  will 
judge  their  merit. 

Before  adopting  achievement  levels  in  a particular  subject  area,  the  Board  examines  relevant  extant 
data  from  other  sources  to  inform  the  decision  and  to  place  it  in  a broader  context.  In  the  case  of  the  1996 
science,  the  Board  examined  other  data  sources,  including  Advanced  Placement  (AP)  science  data  for  the 
grade  12  cohort;  Third  International  Mathematics  and  Science  Study  (TTMSS)  data  for  grade  8;  and 
performance  standards  already  set  by  the  Board  in  reading,  mathematics,  U.S.  history,  and  geography.  They 
also  examined  the  individual  ratings  of  panelists  who  participated  in  the  ACT  level  setting  process. 

In  the  final  analysis,  the  Board  adopted  cut  scores  at  points  on  the  scale  they  judged  to  be 
reasonable,  taking  other  available  information  into  account.  The  interim  cut  points  on  the  NAEP  scale 
varied  from  those  generated  through  the  level  setting  process  in  non-systematic  ways;  some  were  higher 
than  the  recommendations;  some  were  lower.  Consequently,  the  relationships  between  the  interim  cut 
scores  and  the  content  descriptions  was  not  clear. 

Therefore,  the  Board  determined  that  a further  examination  of  the  descriptions  was  necessary,  either 
to  modify  those  that  existed,  or  to  craft  new  ones  that  would  be  more  consistent  with  the  policy  definitions, 
the  framework,  and  the  item  pool  used  to  measure  the  1996  NAEP  science  performance.  In  this  way,  the 
Board  could  have  some  assurance  that  the  descriptions  would  be  providing  a proper  interpretation  of  the 
interim  cut  scores  of  what  students  know  and  can  do  at  the  various  achievement  levels.  This  report  provides 
a summary  of  the  process  used  to  develop  the  descriptions  of  the  levels,  as  well  as  the  results  of  this 
initiative  from  a meeting  convened  in  St.  Louis,  Missouri  from  June  19  through  June  22,  1997. 
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Panel  Selection 


The  plan  called  for  convening  about  24  content  experts  who  have  knowledge  of  what  students  in  the 
respective  grades  can  do  and  are  capable  of  doing  in  elementary,  middle  school,  and  secondary  school 
science.  Generally  teachers,  curriculum  specialists,  and  science  educators  are  in  the  best  position  to  make 
these  kinds  of  judgments.  The  Board-approved  plan  also  called  for  some  number  of  noneducators,  though  a 
proportionally  smaller  number  than  is  typically  used  in  the  level  setting  process.  The  Advisory  Committee 
on  Education  Statistics  (ACES)  who  provides  technical  advice  to  the  Board,  recommended  that  the  grade 
level  groups  include  at  least  5 panelists  (rather  than  4 as  was  suggested  in  the  original  plan). 

A combination  of  two  sources  of  experts  was  used  to  identify  participants:  (1)  states  who  had 
participated  in  the  1996  NAEP  state  assessments;  and  (2)  an  American  College  Testing  (ACT)  sample 
previously  identified  for  the  various  panels  which  ACT  assembles  in  conducting  its  work  for  the  Board 
under  the  achievement  level  setting  contract.  Thirty-two  state  assessment  directors  were  contacted  and 
encouraged  to  submit  the  name  of  one  person  who  would  represent  their  state  at  the  meeting.  Twenty-three 
states  responded  by  electing  to  send  a participant,  with  one  state  (MA)  electing  to  send  two,  and  another 
state  (TX)  sending  three  representatives.  Participating  jurisdictions  are  shown  in  Figure  G-1. 

Figure  G-1 

Participating  Jurisdictions  in  Developing  Achievement  Level 
Descriptions  for  the  1996  NAEP  Science  Assessment 

Georgia  Montana 

Hawaii  Nebraska 

Louisiana  Nevada 

Maine  North  Dakota 

Massachusetts  New  York 

Minnesota  Rhode  Island 


Because  some  states  in  the  initial  sample  were  not  able  to  send  a representative, 
participants  from  a single  state  were  honored.  In  addition,  three  noneducator  panelists  participated,  taken 
from  the  sample  of  nine  nominees  supplied  by  ACT.  The  total  number  of  panelists  was  28.  Participants 
were  given  the  opportunity  to  select  the  grade  level  to  which  they  wished  to  be  assigned  in  a pre-registration 
interview.  They  were  given  their  first  choice  in  all  cases,  and  randomly  assigned  to  one  of  the  two  parallel 
groups  (either  Group  A or  Group  B),  ensuring  that  male  participants  and  noneducators  were  assigned 
throughout  the  groups  (since  there  were  fewer  participants  in  both  categories). 


Alabama 

Alaska 

Arizona 

Connecticut 

Delaware 

District  of  Columbia 


Tennessee 

Texas 

Utah 

Virginia 

Washington 


the  multinle 


Study  Design 

The  plan  approved  by  the  Board  called  for  two  independent  panels  to  conduct  this  work.  Each 
panel  was  composed  of  14  persons,  assigned  to  grade  level  groups  of  4 or  5 persons  per  grade.  All 
participants  were  provided  the  same  training  and  were  asked  to  complete  the  same  tasks.  The  design  of  two 
independent  panels  was  for  purposes  of  embedding  a validity  check  into  the  process,  which,  if  successful, 
would  provide  a built-in  replication  study.  Panelists  were  asked  in  the  opening  plenary  session  to  honor  the 
independent  groups  design  and  to  refrain  from  discussing  their  work  with  members  of  the  other  group  until 
it  was  appropriate  to  do  so,  on  the  last  day. 


The  plan  was  implemented  as  a multi-step  process,  with  individual  judgments  being  the  focus  of 
Day  2,  within-grade  consensus  the  focus  of  Day  3,  and  cross-grade,  cross-level  consistency  being  the  focus 
of  Day  4.  Further,  it  was  made  clear  to  the  panelists  that  this  was  an  empirically-based  design  insofar  as 
data  based  on  actual  student  performances  was  the  basis  for  developing  the  descriptions  of  what  students 
know  and  can  do  at  each  level. 


Materials 

Prior  to  the  meeting  panelists  were  sent  copies  of  the  1996  NAEP  science  assessment  framework. 
They  referred  to  this  regularly  during  the  course  of  their  work.  In  addition,  on  Day  2 they  were  given  the 
item  anchor  book  from  which  they  would  do  all  their  work.  The  items  in  a given  grade  level  appeared  in  the 
anchor  book  in  the  order  in  which  they  map  to  the  NAEP  scale,  from  easiest  to  hardest.  In  addition, 
panelists  were  given  an  index  listing  of  the  items  in  the  same  order  as  the  anchor  book,  copies  of  the  policy 
definitions,  and  forms  on  which  they  could  submit  the  work  of  Day  2 and  Day  3. 


Training 

Panelists  were  trained  simultaneously  to  insure  standardized  exposure  to  all  relevant  information 
needed  to  accomplish  the  task.  Since  all  panelists  were  not  equally  familiar  with  NAEP  and  NAGB,  a brief 
introduction  was  provided  covering  such  topics  as  the  scope  of  the  framework,  the  nature  of  the  item  pool,  a 
non-technical  summary  of  NAEP  scaling  and  item  mapping,  and  instructions  on  how  to  read  the  information 
provided  in  the  anchor  binders  containing  all  the  items,  from  easiest  to  hardest. 


During  the  first  plenary  session,  panelists  were  given  an  overview  of  NAEP  and  NAGB,  and  had 
the  tasks  of  the  next  three  days  presented  in  summary  form.  The  training  focused  on  giving  them  a level  of 
comfort  with  the  tasks  they  were  going  to  be  doing,  introducing  the  scope  of  the  1996  NAEP  science 
assessment,  with  a full  explanation  of  the  content  of  the  assessment,  the  vocabulary  peculiar  to  NAEP,  e.g., 
"blocks  of  exercises,"  and  the  chronology  of  the  1996  NAEP  assessment.  It  was  important  for  panelists  to 
understand  where  they  were  coming  into  the  process,  that  the  assessment  had  already  been  administered, 
scored,  scaled,  and  reported.  The  achievement  levels  remained  the  only  unfinished  aspect  of  the  science 
assessment. 

On  Day  2 the  agenda  called  for  a plenary  session  to  provide  specific  training  in  the  task  of  making 
independent  judgments  about  the  content  represented  by  the  items  at  the  grade  level.  Though  not  described 
in  these  terms,  this  activity  was  designed  to  provide  panelists  with  first-hand  knowledge  of  the  contents  of 
the  NAEP  assessments,  the  kinds  of  items  used,  the  manner  of  scoring  the  items,  and  the  performance  of 
students  on  average,  at  the  borderline  of  Basic,  Proficient,  and  Advanced,  and  across  the  levels.  The  time 
spent  in  reviewing  each  item  without  the  assistance  of  other  panelists  forced  each  participant  to  deal  with 
the  notion  of  item  difficulty  without  being  influenced  by  others,  and  with  the  idea  of  operationalizing  the 
Board’s  policy  definitions.  The  real  goal  of  Day  2,  though  not  formally  stated  to  panelists,  was  to  train  them 
in  the  item  pool.  Requiring  each  panelist  to. make  an  individual  judgment  about  each  item  was  more  likely 
to  have  them  read  and  try  to  understand  the  cognitive  demand  of  the  items  than  would  the  simpler  task  of 
reading  the  items  thoughtfully. 

Panelists  were  directed  to  start  with  the  items  in  the  Basic  level,  those  items  falling  between  the 
lower  and  upper  borderline  for  Basic  (according  to  the  interim  levels  adopted  by  the  Board),  and  generate 
working  lists  of  general  statements  representing  the  content  of  the  assessment.  They  could  generate  these 
lists  by  clustering  similar  content  together  and  developing  generalized  statements  which  summarized  that 
content.  When  they  had  completed  this  part  of  the  task,  they  were  asked  to  compare  the  results  with  the 
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policy  definition  for  Basic,  i.e.,  partial  mastery  of  the  prerequisite  knowledge  and  skill  fundamental  for 
proficient  work.  The  consistency  check  between  the  descriptions  and  the  definitions  was  encouraged  on  an 
holistic  basis,  not  an  item-by-item  basis.  The  question  to  be  answered  was,  "Does  this  description,  in 
general,  reflect  the  policy  definition?",  not  "Does  this  item,  and  this  item,  etc.,  reflect  the  policy 
definition?"  If  the  panelist  was  satisfied  with  the  general  consistency  between  the  two,  they  could  move  on 
to  the  next  level.  Proficient,  and  repeat  the  process;  then  to  Advanced,  and  repeat  the  process  a third  time. 
The  product  for  Day  2 was  a working  list  of  statements  for  each  level  from  each  participant.  The  grade  4 
panelists  reviewed  226  items  in  this  process;  grades  8 and  12  reviewed  312  items  each. 

During  the  training  panelists  were  given  suggestions  for  verbs  that  could  be  used  to  indicate 
increasing  knowledge  and  skills  from  one  level  to  the  next,  as  well  as  common  phrases  that  are  also 
indicators  of  hierarchical  skill  levels  and  increasing  levels  of  sophistication  in  the  knowledge  domain. 

These  linguistic  features  had  been  used  in  the  past  in  other  anchoring  settings  and  were  judged  to  be  useful 
in  this  one  as  well.  Panelists  were  not  required  to  use  these  words,  but  most  found  them  helpful  in 
completing  the  task. 


Panelists  were  also  trained  in  how  to  "read"  the  data  accompanying  each  item.  The  scale  value  for 
the  item  using  a response  probability  (rp)  criterion  of  .65  was  explained,  as  well  as  the  differences  between 
the  "borderline"  p-values  and  the  "across  the  range"  p-values.  The  training  also  focused  on  the  differences 
between  a constructed  response  coded  [2]  or  partial,  and  those  coded  [3]  or  [4],  complete. 

Finally,  panelists  were  led  through  a set  of  about  10  items  as  a model  for  the  task  they  were  about  to 
complete,  demonstrating  how  to  use  the  data,  judge  the  difficulty  of  the  items,  and  write  statements  using 
the  linguistic  suggestions. 


Day  3 was  designed  to  have  the  grade  level  panels  confer  and  reach  group  agreement  on  the  initial 
descriptions  for  Bask,  Proficient,  and  Advanced  at  grades  4,  8,  and  12.  As  on  the  previous  day,  the  plenary 
session  in  the  morning  was  focused  on  the  groups’ task.  They  were  instructed  to  start  by  reading  each 
other’s  lists  from  the  work  of  Day  2.  They  were  then  to  select  those  statements  from  the  lists  for  which  there 
was  general  group  agreement,  and  to  return  to  the  remaining  statements  where  there  was  lack  of  agreement 
and  work  them  through  to  inclusion  or  exclusion,  in  the  end  creating  a group  master  working  list  for  Basic. 
They  needed  to  ensure  that  in  general  the  master  list  was  consistent  with  the  policy  definition  for  Basic, 
before  they  could  move  on  to  the  Proficient  or  Advanced  levels.  Each  group  worked  between  the  individual 
lists  created  the  previous  day  and  flip  charts  of  the  master  lists.  For  tracking  purposes,  groups  were 
encouraged  to  list  the  items  that  were  the  underpinnings  for  their  statements,  thus  creating  a "paper  trail",  as 

it  were,  for  the  statements  in  the  lists.  The  product  for  Day  3 was  a working  list  for  the  three  levels  for  each 
grade  level  group. 


Day  4 training  was  focused  on  achieving  clarity  in  the  statements,  examining  consistency  across 
levels,  across  grades,  and  with  the  policy  definitions.  Panelists  were  organized  conference  style  which 
supported  group  discussion  and  consensus-building.  Panelists  were  asked  to  track  the  general  statements 
found  to  be  inconsistent,  as  well  as  the  associated  items,  so  that  a "paper  trail"  existed  for  the  content  judged 
to  be  irregular  or  inappropriately  placed  at  a particular  level.  Such  anomalies  would  be  handled  at  the  end 
of  the  work  session  if  necessary. 


Results 


The  results  of  the  initiative  can  be  found  in  Figures  G-2,  G-3,  and  G-4  below.  These  are  the 
narrative  versions  of  the  results,  which  have  been  professionally  edited  and  returned  to  the  panelists  for  their 
concurrence.  Table  G-1  shows  the  percentages  of  panelists  who  approved  the  narratives. 
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Figure  G-2 

1996  NAEP  Science  Achievement  Level  Descriptions 
Grade  4 


Cut  Score 

Content  Descrintions*  > 

Basic 

138 

Students  performing  at  the  Basic  level  demonstrate  some  of  the  knowledge  and  reasoning  required  for  understanding  of 
the  earth,  physical,  and  life  sciences  at  a level  appropriate  to  Grade  4,  For  example,  they  can  carry  out  simple 
investigations  and  read  uncomplicated  graphs  and  diagrams.  Students  at  this  level  also  show  a beginning  understanding  of 
classification,  simple  relationships,  and  energy. 

Fourth-grade  students  performing  at  the  Basic  level  are  able  to  follow  simple  procedures,  manipulate  simple  materials, 
make  observations,  and  record  data.  They  are  able  to  read  simple  graphs  and  diagrams  and  draw  reasonable  but  limited 
conclusions  based  on  data  provided  to  them.  These  students  can  recognize  appropriate  experimental  designs,  although  they 
are  unable  to  justify  their  decisions. 

When  presented  with  diagrams,  students  at  this  level  can  identify  seasons;  distinguish  between  day  and  night;  and  place 
the  position  of  the  Earth,  sun,  and  planets.  They  are  able  to  recognize  major  energy  sources  and  simple  energy  changes.  In 
addition,  they  show  an  understanding  of  the  relationship  between  sound  and  vibrations.  These  students  are  able  to  identify 
organisms  by  physical  characteristics  and  group  organisms  with  similar  physical  features.  They  can  also  describe  simple 
relationships  among  structure,  function,  habitat,  life  cycles,  and  different  organisms. 

Proficient 

170 

Students  performing  at  the  Proficient  level  demonstrate  the  knowledge  and  reasoning  required  for  understanding  ol  the 
earth,  physical,  and  life  sciences  at  a level  appropriate  to  Grade  4.  For  example,  they  understand  concepts  relating  to  the 
Earth’s  features,  physical  properties,  and  structure  and  function.  In  addition,  student  can  formulate  solutions  to  familiar 
problems  as  well  as  show  a beginning  awareness  of  issues  associated. with  technology. 

Fourth-grade  students  performing  at  the  Proficient  level  are  able  to  provide  an  explanation  of  day  and  night  when  given  a 
diagram.  They  can  recognize  major  features  of  the  Earth’s  surface  and  the  impact  of  natural  forces.  They  are  also  able  to 
recognize  water  in  its  various  forms  in  the  water  cycle  and  can  suggest  ways  to  conserve  it.  These  students  recognize  that 
various  materials  possess  different  properties  that  make  them  useful.  Students  at  this  level  are  able  to  explain  how 
structure  and  function  help  living  things  survive.  They  have  a beginning  awareness  of  the  benefits  and  challenges 
associated  with  technology  and  recognize  some  human  effects  on  the  environment.  They  can  also  make  straightforward 
predictions  and  justify  their  position. 

Advanced 

204 

Students  performing  at  the  Advanced  level  demonstrate  a solid  understanding  of  the  earth,  physical,  arid  life  sciences  as 
well  as  the  abilitv  to  apply  their  understanding  to  practical  situations  at  a level  appropriate  to  Grade  4.  For  example,  they 
can  perform  and  critique  simple  investigations,  make  connections  from  one  or  more  of  the  sciences  to  predict  or  conclude, 
and  applY  fundamental  concepts  to  practical  applications. 

Fourth-grade  students  performing  at  the  Advanced  level  are  able  to  combine  information,  data,  and  knowledge  from  one 
or  more  of  the  sciences  to  reach  a conclusion  or  to  make  a valid  prediction.  They  can  also  recognize,  design,  and  explain 
simple  experimental  procedures. 

Students  at  this  level  recognize  nonrenewable  sources  of  energy.  They  also  recognize  that  light  and  sound  travel  at 
different  speeds.  These  students  understand  some  principles  of  ecology  and  are  able  to  compare  and  contrast  life  cycles  of 
various  common  organisms.  In  addition,  they  have  a developmental -awareness  of  the  benefits  and  challenges  associated 
with  technology. 

♦Shaded  areas  indicate  summary  of  content  descriptions. 
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Figure  G-3 

1996  NAEP  Science  Achievement  Level  Descriptions 
Grade  8 


Cut  Score 

- — ■ Content  Descriptions 

Basic 

143 

Students  performing  at  the  Basic  level  demonstrate  some  of  the  knowledge  and  reasoning  required  for  understandinrc* 
the  earth,  physical,  tmd  life  sconces  at  a level  appropriate  to  Grade  8.  For  example,  they  can  cany  out  investigations  alii 
obtain  information  from  graphs,  diagrams,  and  tables.  In  addition,  they  demonstrate  soiheluhderst^^^  of  mm 

also  have  a beginnin|  understanding  of  caus^^ 

Eighth-grade  students  performing  at  the  Basic  level  are  able  to  observe,  measure,  collect,  record,  and  compute  data  from^ 
investigations.  They  can  read  simple  graphs  and  tables  and  are  able  to  make  simple  data  comparisons.  These  students  are 
able  to  follow  directions  and  use  basic  science  equipment  to  perform  simple  experiments.  In  addition,  they  have  an 
emerging  ability  to  design  experiments. 

Students  at  this  level  have  some  awareness  of  causal  relationships.  They  recognize  the  position  of  planets  and  their 
movement  around  the  sun  and  know  basic  weather-related  phenomena.  These  students  can  explain  changes  in  position  and 
motion  such  as  the  movement  of  a truck  in  relation  to  that  of  a car.  They  also  have  an  emerging  understanding  of  the 
interrelationships  among  plants,  animals,  and  the  environment. 

ProHcient 

170 

Students  performing  at  the  Proficient  level  demonstrate  much  of  the  knowledge  and  many>:of  the  reasonings  abilitii^; 
essential  for  understanding  of  the  earth,  physical,  and  life  sciences  at  a level  appropriate  to  Grade  8:  For  example^  students! 
can  interpret  graphic  information,  design  simple  investigations,  and  fexplMh  such:  scientiricfcphcept^^ 

Students  at  this  level  also  show  an  awareness  of  environmenj^l  issues,  especially  those  addressing  energy  and  pollutibn.O-^ 
Eighth-grade  students  performing  at  the  Proficient  level  are  able  to  create,  interpret,  and  make  predictions  from  charts, 
diapams,  and  graphs  based  on  information  provided  to  them  or  from  their  own  investigations.  They  have  the  ability  to 
design  an  experiment  and  have  an  emerging  understanding  of  variables  and  controls.  These  students  are  able  to  read  and 
interpret  geographic  and  topographic  maps.  In  addition,  they  have  an  emerging  ability  to  use  and  understand  models,  can 
partially  formulate  explanations  of  their  understanding  of  scientific  phenomena,  and  can  design  plans  to  solve  problems. 

Students  at  this  level  can  begin  to  identify  forms  of  energy  and  describe  the  role  of  energy  transformations  in  living  and 
nonliving  systems.  They  have  knowledge  of  organization,  gravity,  and  motions  within  the  solar  system  and  can  identify 
some  factors  that  shape  the  surface  of  the  Earth.  These  students  have  some  understanding  of  properties  of  materials  and 
have  an  emerging  understanding  of  the  particulate  nature  of  matter,  especially  the  effect  of  temperature  on  states  of  matter. 
They  also  know  that  light  and  sound  travel  at  different  speeds  and  can  apply  their  knowledge  of  force,  speed,  and  motion. 
These  students  demonstrate  a developmental  understanding  of  the  flow  of  energy  from  the  sun  through  living  systems, 
especially  plants.  They  know  that  organisms  reproduce  and  that  characteristics  are  inherited  from  previous  generations. 
These  students  also  understand  that  organisms  are  made  up  of  cells  and  that  cells  have  subcomponents  with  different 
functions.  In  addition,  they  are  able  to  develop  their  own  classification  system  based  on  physical  characteristics.  These 
students  can  list  some  effects  of  air  and  water  pollution  as  well  as  demonstrate  knowledge  of  the  advantages 
Students  performing  at  the  Advanced  level  demonstrate  a solid  understanding  of  the  earth;  physical,  and  life  sciences  as 
well  as  the  abilities  required  to  apply  their  understanding  in  practical  situations  at  a level  appropriate  to  Grade  8.  For 
example,  students  perform  and  critique  the  desijgn  of  investigations,  relate  ;scientific  concepts;  to 'each  other,  explain  their 
reasoning,  and  discuss  the  impact  of  human  activities  on  the  environment.  z:  • ! . : 

Advanced 

207 

Eighth-grade  students  performing  at  the  Advanced  level  are  able  to  provide  an  explanation  for  scientific  results.  They  have 
a modest  understanding  of  scale  and  are  able  to  design  a controlled  experiment.  These  students  have  an  understanding  of 
models  as  representations  of  natural  systems  and  can  describe  energy  transfer  in  living  and  nonliving  systems. 

Students  at  this  level  are  able  to  understand  that  present  physical  clues,  including  fossils  and  geological  formations,  are 
indications  that  the  Earth  has  not  always  been  the  same  and  that  the  present  is  a key  to  understanding  the  past.  They  have  a 
solid  Imowledge  of  forces  and  motions  within  the  solar  system  and  an  emerging  understanding  of  atmospheric  pressure. 
These  students  can  recognize  a wide  range  of  physical  and  chemical  properties  of  matter  and  some  of  their  interactions  and 
understand  some  of  the  properties  of  light  and  sound.  Also,  they  can  infer  relationships  between  structure  and  function. 
These  students  know  the  differences  between  plant  and  animal  cells  and  can  apply  their  knowledge  of  food  as  a source  of 
energy  to  a practical  situation.  In  addition,  they  are  able  to  explain  the  impact  of  human  activities  on  the  environment  and 
the  economy. 

♦Shaded  areas  indicate  summary  of  content  descriptions. 
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Figure  G-4 

1996  NAF.P  Science  Achievement  Level  Descriptions 
Grade  12 


Cut  Score 


Content  Descrintions'*' 


Basic 

145 


Proficient 

178 


Students  performing  at  the  Basic  level  demonstrate  some  knowledge;  and  certain  reasoning  abilities  required  for 
understanding  of  the  earth,  physical,  and  life  sciences  at  a level  appropriate  to  Grade  12..to,a^  they  demonstrate.] 
knowledge  of  the  themes  of  science  (models,  systems,  patterns  :of;  change)  required  for  understanding  the  most  basic  ' 
relationships  among  the  earth,  physical,  and  life  sciences.  They  are-able  to  conduct  investigations,  critique  the  design  of 

investigations,  and  demonstrate  a rudimentary  understanding  of  scientific  principles.  ---i:  

Twelfth-grade  students  performing  at  the  Basic  level  are  able  to  select  and  use  appropriate  simple  laboratory  equipment 
and  write  down  simple  procedures  that  others  can  follow.  They  also  have  a developmental  ability  to  design  complex 
experiments.  These  students  are  able  to  make  classifications  based  on  definitions  such  as  physical  properties  and 
characteristics. 

Students  at  this  level  demonstrate  a rudimentary  understanding  of  basic  models  and  can  identify  some  parts  of  physical 
and  biological  systems.  They  are  also  able  to  identify  some  patterns  in  nature  and  rates  of  change  over  time.  These  students 
have  the  ability  to  identify  basic  scientific  facts  and  terminology  and  have  a rudimentary  understanding  of  the  scientific 
principles  underlying  such  phenomena  as  volcanic  activity,  disease  transmission,  and  energy  transformation.  In  addition, 

they  have  some  familiarity  with  the  application  of  technology. ^ ^ 

Stullents  performing  at  the  Proficient  level  demonstrate  the  knowledge  and  reasoning  abiUties  required  for  understanaing 
of  the  earth,  physical,  and  life  sciences  at  a level  appropriate  to  Grade.-12.  In  addition,  they  demonstrate  knowledge  ot  the 
themes  of  science  (models,  systems,  patterns  of  change)  requiredfor  understanding  how  these  themes  illustrate  essential 
relationships  among  the  earth,  physical,  and.  life  sciences;  They  arei^jle  to  analyze  data  and  apply  scientific  pnnciples  to 
even'clay  situations. 


^ , ■ ....  II  

Tvvelfth-grade  students  performing  at  the  Proficient  level  are  able  to  demonstrate  a working  ability  to  design  and  conduct 
scientific  investigations.  They  are  able  to  analyze  data  in  various  forms  and  utilize  information  to  provide  explanations  and 
to  draw  reasonable  conclusions. 

Students  at  this  level  have  a developmental  understanding  of  both  physical  and  conceptual  models  and  are  able  to  compare 
various  models.  They  recognize  some  inputs  and  outputs,  causes  and  effects,  and  interactions  of  a system.  In  addition,  they 
can  correlate  structure  to  function  for  the  parts  of  a system  that  they  can  identify.  These  students  also  recognize  that  rate  of 
change  depends  on  initial  conditions  and  other  factors.  They  are  able  to  apply  scientific  concepts  and  principles  to 
practical  applications  and  solutions  for  problems  in  the  real  world  and  show  a developmental  understanding  of  technology, 
its  uses,  and  its  applications. 


Advanced 

210 


Students  performing  at  the  Advanced  level  demonstrate  the  knowledge  and  reasoning:  abilities  required  for  a solid 
understanding  of  the  earth,  physical,  and  life  sciences  at  a . level  appropriate  to  Grade.  12:  In  addition,  they  demonstrate 
knowledge  of  the  themes  of  science  (models,  systems,  patterns:  of  change)  required  "for.  integrating  knowledge  and 
understanding  of  scientific  principles  from  the  earth,  physical,  and  life  sciences.  Students: ciwi  design  investigations  that 
answer  questions  about  real-world  situations  and  use  their  reasoning  abilities  to  make  predications. 

Twelfth-grade  students  performing  at  the  Advanced  level  are  able  to  design  scientific  investigations  to  solve  complex, 
real-world  situations.  They  can  integrate,  interpolate,  and  extrapolate  information  embedded  in  data  to  draw  well- 
formulated  explanations  and  conclusions.  They  are  also  able  to  use  complex  reasoning  skills  to  apply  scientific  knowledge 
to  make  predictions  based  on  conditions,  variables,  and  interactions. 

Students  at  this  level  recognize  the  inherent  strengths  and  limitations  of  models  and  can  revise  models  based  on  additional 
information.  They  are  able  to  recognize  cause-and-effect  relationships  within  systems  and  can  utilize  this  knowledge  to 
make  reasonable  predictions  of  future  events.  These  students  are  able  to  recognize  that  patterns  can  be  constant, 
exponential,  or  irregular  and  can  apply  this  recognition  to  make  predictions.  They  can  also  design  a technological  solution 
for  a given  problem. 


♦Shaded  areas  indicate  summary  of  content  descriptions. 
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Table  G-1 

Percentage^  of  Panelists  Approving 
the  Edited  Versions  of  the  Descriptions 
(Narrative  Format) 


Grade  4 

Grade  8 

Grade  12 

Group  A 

100% 

100% 

67% 

Group  B 

100 

100 

100 

Total 

100 

100 

80 

The  descriptions  have  been  reviewed  by  the  three  science  specialists  who  facilitated  the  meeting.  In 
general,  there  are  few  differences  to  be  noted  between  the  results  for  Group  A and  Group  B,  though, 
according  to  one  reviewer.  Group  B’s  statements  tend  to  be  more  general  than  Group  A’s.  Even  though  the 
descriptions  were  produced  on-site,  without  benefit  of  time  to  be  very  selective  in  language  and  style,  and 
the  opportunity  to  conduct  careful  checks  across  grades  and  levels,  they  are  remarkably  consistent  one  to 
another.  However,  the  consensus  among  reviewers  is  that  Group  A’s  descriptions  are  probably  more 
consistent  both  within  and  across  grades  than  Group  B’s,  as  well  as  more  complete  in  terms  of  representing 
the  items  in  the  pool  than  Group  B’s. 


Evaluation 

Panelists  were  asked  to  complete  an  evaluation  questionnaire  at  the  end  of  the  process.  The 
purpose  of  the  evaluation  was  to  accomplish  two  goals:  (1)  provide  feedback  regarding  the  panelists’ 
opinion  about  the  consistency  of  the  achievement  levels  descriptions  across  levels  and  grades,  as  well  as 
between  the  descriptions  and  the  policy  definitions;  and  (2)  provide  feedback  on  the  interim  cut  scores  and 
the  percentage  of  students  at  or  above  the  levels.  The  questions  covering  the  latter  were  adapted  from  an 
evaluation  questionnaire  used  by  ACT  in  conducting  the  achievement  level  setting  process.  It  should  be 
noted  that  the  Board-approved  plan  did  not  allow  for  providing  consequences  data  to  the  panelists  during 
the  working  sessions.  However,  consequences  data  were  presented  on  the  evaluation  questionnaire  during 
the  debriefing  segment  of  the  meeting  when  it  could  in  no  way  influence  the  results. 

The  first  eight  questions  dealt  with  consistency.  Panelists  provided  a rating  from  [\\not  at  all 
consistent,  to  [5]  very  consistent.  In  examining  differences  between  the  two  parallel  groups,  [A]  and  [B], 
Group  [B]  reports  slightly  more  consistency  than  does  Group  [A].  Within  Group  [A],  the  grade  12  group 
reports  less  consistency  in  all  comparisons  than  either  grades  4 or  8. 

The  questionnaire  also  asked  a series  of  questions  which  were  adapted  from  the  ACT 
questionnaires  regarding  the  cut  scores  adopted  on  an  interim  basis,  the  percentage  of  students  at  or  above 
the  levels,  and  the  achievement  levels  descriptions.  Additionally,  the  questionnaire  gave  panelists  the 
opportunity  to  recommend  no  changes  in  the  cut  scores  and  thus  leaving  the  percentages  of  students  who 
score  at  or  above  the  levels  as  presented,  or  to  recommend  smaller  or  larger  percentages  of  students  who 


* Percentages  based  on  the  total  number  of  respondents,  not  necessarily  the  total  number  of  panelists. 


O .66 

ERIC 


S50 


should  score  at  or  above  the  levels  and  consequently  changing  the  cut  scores.  Opinion  was  split  evenly 
among  those  who  would  not  change  the  cut  scores  and  those  who  would  either  raise  or  lower  the  cut  scores. 

The  final  question  asked  panelists  to  take  a position  regarding  the  recommendations  they  wished  to 
make  to  the  National  Assessment  Governing  Board  regarding  the  achievement  level  descriptions.  The 
majority  of  panelists  (between  64%  and  71%)  recommended  that  the  achievement  levels  adopted  by  the 
Board  on  an  interim  basis,  and  which  were  used  to  develop  the  achievement  levels  descriptions,  be  reported. 


Summary  and  Recommendations 

The  Board’s  plan  called  for  describing  the  performance  of  students  within  the  ranges  on  the  NAEP 
scale  for  the  achievement  levels  adopted  by  the  Board  on  an  interim  basis.  This  report  provides  the  results 
of  the  Board’s  plan  as  carried  out  by  this  process.  These  descriptions  represent  what  students  know  and  can 
do  within  the  achievement  level  ranges  of  Basic,  Proficient,  and  Advanced. 

It  is  recommended  that,  should  the  Board  adopt  the  interim  levels  on  a permanent  basis  for  the  1996 
NAEP  science  achievement  levels,  then  these  descriptions  should  be  adopted  to  assist  in  the  proper 
interpretation  of  the  NAEP  scores.  It  is  also  recommended  that  in  the  Board’s  report  on  the  levels,  the 
differences  between  these  descriptions  on  the  1996  NAEP  science  assessment  of  what  students  know  and 
can  do  and  achievement  levels  descriptions  in  other  NAEP  content  areas  of  what  students  should  know  and 
be  able  to  do,  needs  to  be  made  clear  to  the  readers.  Additionally,  it  is  recommended  that  the  exemplar 
items  (in  a separate  document)  which  are  typical  of  student  performances  for  each  level  accompany  the  cut 
scores  and  descriptions  to  assist  in  the  proper  interpretation  of  the  NAEP  scores. 

Finally,  a word  about  the  Group  A and  Group  B descriptions.  The  purpose  of  the  parallel  working 
groups  was  to  provide  the  Board  with  some  evidence  that  the  resulting  descriptions  were  not  idiosyncratic  to 
a particular  set  of  panelists.  The  evidence  points  to  the  fact  that  either  set  of  descriptions  could  be  used  in 
the  report.  However,  since  the  Group  A descriptions  seem  to  be  slightly  more  internally  consistent,  it  is 
recommended  that  Group  A’s  become  the  official  set,  and  Group  B’s  be  viewed  as  the  validation  set,  and  not 
used  to  report  the  achievement  levels  results. 
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Appendix  H 


THE  INFORMATION  WEIGHTING  ERROR 


Susan  C.  Loomis,  Luz  Bay,  and  Wen-Hung  Chen 
American  College  Testing 


The  Error 


In  the  process  of  recomputing  the  reading  cutscores  set  in  1992  for  the  three  achievement 
levels,  an  error  in  the  information  weighting  function  was  detected.  The  error  affected  data  for  all 
achievement  levels  set  in  1992:  reading  and  mathematics.  The  Muraki  information  weighting 
function  published  in  1993  was  used  in  the  1994  programs  to  compute  achievement  levels,  so 
only  1992  levels  are  affected. 

The  procedures  used  for  1992  were  printed  and  reported  in  numerous  places.  No  one  had 
detected  an  error.  The  psychometrician  who  developed  the  programs  for  the  1994  process  used 
Muraki’ s information  weighting  function  because  he  found  it  to  be  more  straightforward  than  the 
1992  procedure. 

The  1992  equation'  is  as  follows: 


Ij(d)  = 


The  1994  equation  is  . as  follows: 


Ij  (d)  = aj  ^[T,-Tj( 0)f  Pjc  ( 6)  , 


C=1 


where  T is  the  expected  score  for  item  j or  proficiency  6 


and  Tc  is  the  score  assigned  to  the  response  category  c. 


^ Notations  for  this  equation  were  modified  to  correspond  to  those  of  Equation  2.  The  reader  will  need  to  refer  to  the 
articles  (Luecht,  1993,  and  Muraki,  1993)  for  a complete  explanation  of  the  equations. 
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Analysis  of  the  Error:  Magnitude 

The  differences  in  achievement  levels  reported  for  1992  and  the  corrected  achievement 
levels  are  due  both  to  the  error  in  item  parameters  and  to  the  error  in  information  weights.  The 
cutscores  and  percentages  of  students  scoring  at  or  above  each  for  each  achievement  level  are 
reported  in  Tables  H-1  and  H-2.  Data  in  Table  H-1  are  the  previously  reported  (incorrect)  data, 
and  data  in  Table  H-2  are  the  corrected  data. 

Table  H-1 

Mathematics  Cutpoints  and  Percents  At  or  Above  as  Reported 


Grade 

Basic 

Proficient 

Advanced 

4 

Cutpoint 

211 

248 

280 

%>92  Dist 

61 

18 

2 

8 

Cutpoint 

256 

294 

331 

%>92  Dist 

63 

25 

4 

12 

Cutpoint 

287 

334 

366 

%>92  Dist 

64 

16 

2 

Table  H-2 

Corrected  Mathematics  Cutpoints  and  Percents  At  or  Above  as 

Reported 

Grade 

Basic 

Proficient 

Advanced 

4 

Cutpoint 

214 

249 

282 

%>92  Dist 

59 

18 

2 

8 

Cutpoint 

262 

299 

333 

%>92  Dist 

58 

21 

3 

12 

Cutpoint 

288 

336 

367 

%>92  Dist 

64 

15 

2 

The  corrected  cutscores  are  consistently  the  same  or  higher  than  those  previously 
reported.  The  maximum  difference  in  cutscores  originally  reported  and  the  corrected  cutscores  is 
found  for  grade  8 at  the  Basic  level: 

((original  cutscore  = 256)  - (corrected  cutscore  = 262))  = -6  points. 

The  differences  attributable  to  each  error  (parameter  estimates  and  information  weighting) 
appear  to  be  rather  small  in  most  cases. 

Table  H-3  reports  the  differences  in  cutscores  due  to  the  two  errors,  examined  one  at  a 
time.  Relative  to  the  correct  data,  the  information  weighting  error  generally  resulted  in  a lower 
composite  cutscore,  and  the  recoding  error  resulting  in  incorrect  item  parameters  generally 
resulted  in  an  even  lower  composite  cutscore. 
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Table  H-3 

Composite  NAEP  Scale  Cutpoint  Differences  in  Mathematics  Due  to  Errors 


Achievement  Level  Cutpoint 

Information  Weighting* 

Item  Parameters^ 

Grade  4 

Basic 

-2 

-5 

Proficient 

-1 

-1 

Advanced 

-1 

-1 

Grade  8 

Basic 

-4 

-8 

Proficient 

-3 

-2 

Advanced 

-1 

-2 

Grade  12 

Basic 

0 

-4 

Proficient 

-1 

-2 

Advanced 

-1 

-2 

' Difference  = Incorrect  - Correct,  based  on  correct  item  parameters.  If  the  recoding  of  data  had  been  correct,  the 
cutpoints  would  have  been  in  error  by  these  amounts,  due  to  the  incorrect  information  weighting  function. 

^ Difference  = Incorrect  - Correct,  based  on  correct  information  weights.  If  the  correct  information  weighting  function 
had  been  used,  the  cutpoints  would  have  been  in  error  by  these  amounts  due  to  the  recoding  error  resulting  in  incorrect 
item  parameters. 


Table  H-4  shows  comparisons  of  percentages  of  students  who  scored  at  or  above  each 
achievement  level  in  1992.  The  center  row  for  each  grade  shows  student  performance  relative  to 
each  achievement  level  in  1992  using  both  correct  item  parameters  and  correct  information 
weights.  The  first  row  for  each  grade  shows  student  performance  relative  to  achievement  levels 
computed  with  the  correct  item  parameters  and  incorrect  information  weights.  The  third  row  for 
each  grade  shows  student  performance  relative  to  achievement  levels  computed  with  the 
incorrect  item  parameters  and  the  correct  information  weights. 
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Table  H-4 

NAEP  Mathematics  Achievement  Levels: 
Cutpoints  and  1992  Distribution  Data 


GRADE  4 


Cutpoint 

Percent  At  or  Above  Cutpoint 

Basic 

Proficient 

Advanced 

Basic 

Proficient 

Advanced 

Correct  Data,  Incorrect  Weight 

212 

248 

281 

■ 62.2 

19.7 

2.1 

Correct  Data,  Correct  Weight 

214 

249 

282 

60 

18.7 

1.9 

Incorrect  Data,  Correct  Weight 

209 

248 

281 

65.6 

19.7 

2.1 

GRADES 

Cutpoint 

Percent  At  or  Above  Cutpoint 

Basic 

Proficient 

Advanced 

Basic 

Proficient 

Advanced 

Correct  Data,  Incorrect  Weight 

258 

296 

332 

62.6 

24.4 

3.6 

Correct  Data,  Correct  Weight 

262 

299 

333 

58.6 

21.8 

3.3 

Incorrect  Data,  Correct  Weight 

254 

297 

331 

66.5 

23.5 

3.8 

GRADE  12 

Cutpoint 

Percent  At  or  Above  Cutpoint 

Basic 

Proficient 

Advanced 

Basic 

Proficient 

Advanced 

Correct  Data,  Incorrect  Weight 

288 

335 

366 

64.8 

I6.I 

1.9 

Correct  Data,  Correct  Weight 

288 

336 

367 

64.8 

15.4 

1.8 

Incorrect  Data,  Correct  Weight 

284 

334 

364 

68.7 

16.9 

2.3 

Analysis  of  the  Error:  The  Information  Weighting  Functions 

Various  analyses  were  conducted  to  determine  what,  if  any,  general  conclusions  could  be 
drawn  to  help  inform  users  of  NAEP  achievement  levels  data  about  the  factors  related  to 
differences  in  cutscores  due  to  the  information  weighting  error. 

Item  ratings  are  collected  from  two  groups  of  panelists  at  each  grade  level.  These  groups 
are  called  item  rating  groups,  and  panelists  are  assigned  to  an  item  rating  group  so  that  the  two 
are  as  equivalent  as  possible  in  terms  of  panelist  type  (teacher,  career  educator,  or  general  public; 
gender;  race/ethnicity;  and  region  of  residence).  These  item  rating  groups  rate  slightly  over  half 
of  all  items  at  their  grade  level.  Item  rating  pools  are  developed  so  that  the  items  in  each  are  as 
equivalent  as  possible  in  terms  of  item  difficulty,  item  format  (multiple  choice,  short  constructed 
response,  and  extended  constructed  response),  test  time  for  the  block,  and  so  forth.  Item  blocks 
remain  intact  for  the  item  rating  pools.  At  least  one  block  (a  “common  block”)  is  rated  by  all 
panel  members  (i.e.,  both  item  rating  groups,  in  the  grade  group). 

Item  ratings  are  placed  on  the  NAEP  scale  by  computing  a theta  value  for  the 
dichotomous  items  and  for  the  polytomous  items  in  each  subscale  for  each  rating  group. 
Information  weights  are  applied  for  the  polytomous  items  at  the  subscale  level  before  computing 
the  subscale  score  for  both  dichotomous  and  polytomous  items. 
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Table  H-5  presents  the  information  weights  computed  for  each  rating  group  for  the 
mathematics  NAEP  achievement  levels.  Those  data  show  that  there  is  a consistent  pattern  of 
error  caused  by  the  incorrect  information  function  for  math.  Analyses  of  the  data  from  1992  and 
1994  Trial  State  Assessments  in  reading  show  the  corrected  cutscores  are  consistently  neither 
higher  nor  lower  as  a result  of  this  error,  although  the  impact  of  the  error  was  generally  to 
estimate  a higher  cutscore  for  the  polytomous  items.  Simulated  item  parameters  were  used  to 
demonstrate  that  there  was  no  consistent  effect  of  the  incorrect  information  function  on  weights 
used  to  form  the  final  composite  outpoints. 


Analysis  of  the  Effect  of  Item  Discrimination 

Figures  H-1  through  H-3  show  graphs  for  the  correct  and  the  incorrect  information 
functions  holding  other  parameters  constant  while  varying  the  item  discrimination  parameter  in 
the  generalized  partial  credit  item  response  theory  (IRT)  model.  In  general,  the  differences 
between  correct  and  incorrect  weights  increase  as  item  discrimination  increases. 

These  figures  show  no  consistent  pattern  in  the  direction  (positive  or  negative)  of  the 
difference  between  the  correct  and  incorrect  information  function  based  on  item  discrimination. 
When  the  correct  information  function  is  greater  than  the  incorrect  function,  we  observe  that 
information  is  in  the  area  of  maximum  information.  It  is  not  the  case,  however,  that  the  correct 
function  always  results  in  a greater  information  weight  where  information  is  maximized. 


773 


(DO 

in 

oo 


>> 


u 

C 

CQ 

> 


Q 


o 

U 


<N 

O 


cs 

o 


o 

CN 


00 

d 

I 


d 

I 


o 

d 


o 


r- 

m 

d 

I 


o 

o 


VO 

d 

I 


(N 

00 


ON 

m 

d 

I 


00 

d 


VO 

d 

I 


Ov 


<N 

d 

I 


ov 

VO 

d 


VO 

d 

I 


ov  VO 
<N  <N 

d d 


ov 

p 

cn 

I 


r- 


cn 

00 


VO 

cn 


d 


<N 

d 


o 

m 

d 


<N 

<N 

d 


o 

d 


m 

VO 


o 


ov 

m 

d 


m 

00 

d 


<N 

d 


m 

d 


d 


VO 


Q 


00  <N  ov  Cv> 

^ <N  CO 

d d d d d 

I I I I I 


00 

d 

I 


<N 

as 

d 

I 


VO 

d 

I 


<N 


O 

VO 


VO 

d 

I 


o 

Cn|  ^ 

d CO 

I I 


CO 


as 

p 

CN 

I 


C 


0) 

•mm 

o 


o 


PU 


0> 

o 

U 


0> 

u 

o 

u 

c 


o 

CO 

VO 

o 

OV 

CN 

as 

VO 

CN 

O 

CO 

VO 

yn 

ON 

yn 

p 

ov 

CN 

CO 

p 

SO 

d 

d 

d 

d 

d 

CN 

c4 

d 

d 

d 

CO 

CN 

CN 

»o 

ov 

VO 

00 

CN 

CO 

VO 

ov 

CN 

CN 

CN 

CO 

CN 

p 

ov 

CO 

p 

yn 

d 

d 

d 

d 

d 

d 

d 

d 

d 

d 

d 

Q 


d 

I 


VO 

d 

i 


(N 

d 

i 


o 

d 

I 


as 

d 

I 


yn 

d 

I 


d 

I 


»n 

d 

I 


o 

as 

d 

> 


VO 

d 

I 


CN 

d 

I 


CN 

yn 

CN 

I 


as 


01)  a 
.5  =* 
^ 2 


o 


m < m 


< m 


00 


CN 


LO 

oo 


Figure  H-1 

Comparison  Between  the  Incorrect  and  Correct  Information  Functions 
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Figure  H-2 
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Figure  H-3 

tmparison  Between  the  Incorrect  and  Correct  Information  Functions 
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Analysis  of  the  Effect  of  Location  Parameters 

Figures  H-4  through  H-6  show  the  correct  and  incorrect  information  weighting  functions 
for  varying  location  parameters.  The  location  parameters  only  shift  the  distribution  of 
information,  and  that  is  the  case  for  both  the  correct  and  incorrect  information  weighting 
functions.  The  amount  of  difference  between  the  two  is  unchanged;  only  the  locations  change. 

Analysis  of  the  Effect  of  the  Threshold  Parameters. 

If  the  threshold  parameters  are  close,  in  terms  of  the  locations  of  ICCs,  the  correct 
information  function  will  have  a high  peak.  In  the  area  of  maximum  information,  i.e.  around  the 
peak  of  the  distribution,  the  incorrect  information  function  underestimates  information. 

As  can  be  seen  in  Figures  H-6  through  H-8,  when  the  threshold  parameters  are  relatively 
far  apart,  the  information  is  relatively  low  and  the  distribution  is  multimodal.  When  the  threshold 
parameters  are  closer  information  is  higher  and  the  distribution  tends  to  be  more  unimodal.  As 
the  threshold  parameters  move  even  closer,  the  difference  between  the  correct  and  incorrect 
information  functions  increases.  As  the  threshold  parameters  become  closer,  the  rate  at  which  the 
incorrect  weighting  function  increments  weights  at  the  peak  of  the  distribution  is  slower  than  that 
for  the  correct  function.  This  results  in  a negative  difference  between  the  two  functions  in  the 
area  of  maximum  information. 


Analysis  of  the  Error:  Conclusions 

Three  general  conclusions  can  be  drawn  from  our  analyses: 

1 . No  consistent  pattern  of  over-  or  underestimation  can  be  predicted  from  the 
error  in  information  weights.  Generally,  the  incorrect  information  weighting 
function  results  in  a lower  information  weight  for  1992  mathematics 
assessment  items,  whereas,  it  results  a higher  information  weight  for  1992 
reading  assessment  items. 

2.  The  difference  between  the  correct  and  incorrect  weights  increases  as  item 
discrimination  increases. 

3.  The  impact  of  the  incorrect  information  weighting  function  on  the  cutscores 
is  not  consistent.  The  impact  depends  upon  the  location  of  the  cutscore  and 
the  relative  weight  of  the  dichotomous  items. 
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Figure  H-4 
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Figure  H-5 


CT> 

CO 

CO 


o 

ERIC 


Theta 


Figure  H-6 


00 

CO 


Theta 


Figure  H-8 

Incorrect  and  Correct  Information  Functions 
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Appendix  I 

CONSTRUCTED-RESPONSE  ITEM  SCORE  STATISTICS 


This  appendix  contains  information  about  the  constructed-response  items  included  in  the  scaling  of 
data  from  the  1996  main  assessments  of  mathematics  and  science  and  the  long-term  trend  assessments  of 
reading,  writing,  and  mathematics.  There  were  no  constructed-response  items  included  in  the  scaling  for  the 
1996  long-term  trend  assessment  of  science. 

The  information  in  the  tables  includes,  for  each  subject  area  and  grade  (in  the  case  of  long-term  trend, 
each  age  class),  the  NAEP  item  numbers  for  each  of  the  constructed-response  items  included  in  scaling,  and  the 
block  that  contains  the  item.  The  tables  also  indicate  the  codes  from  the  NAEP  database  that  denote  the  range 
of  responses  and  the  correct  responses  where  appropriate.  A portion  of  the  responses  to  the  constructed- 
response  items  were  scored  twice  for  the  purpose  of  examining  rater  reliability.  For  each  item,  the  number  of 
papers  with  responses  that  were  scored  a second  time  is  listed,  along  with  the  percent  agreement  between  raters 
and  an  index  of  reliability  based  on  those  responses.  Cohen’s  Kappa  (Cohen,  1968)  is  the  reliability  estimate 
used  for  dichotomized  items.  For  items  that  are  not  dichotomized  (i.e.,  polytomous  items),  the  intraclass 
correlation  coefficient  is  used  as  the  index  of  reliability.  See  Chapter  9 for  more  information  about  score 
reliability  for  constructed-response  items. 
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Table  M 

Score  Range,  Percent  Agreement,  and  Cohen’s  Kappa' 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  4^ 


Item 

Block 

Range  of 
Response 
Codes 

M019701 

MF 

1-2 

M019801 

MF 

1-3 

M019901 

MF 

1-3 

M020001 

MF 

1-2 

M020101 

MF 

1-2 

M020201 

MF 

1-2 

M020301 

MF 

1-4 

M020401 

MF 

1-2 

M020501 

MF 

1-2 

M020701 

MF 

1-4 

M039201 

MC 

1-2 

M039301 

MC 

1-3 

M040001 

MC 

1-3 

M040201 

MC 

1-2 

M040301 

MI 

1-2 

M040901 

MI 

1-2 

M043201 

MM 

1-2 

M043301 

MM 

1-3 

M043401 

MM 

1-4 

M043402 

MM  . 

1-4 

M043403 

MM 

1-3 

M046001 

MK 

1-5 

M046601 

MK 

1-4 

M046801 

MK 

1-5 

M046901 

MK 

1-5 

M047301 

MK 

1-4 

M061901 

MI 

1-3 

M061902 

MI 

1-3 

M061903 

MI 

1-2 

M061904 

MI 

1-3 

M061905 

MI 

1-4 

M061906 

MI 

1-3 

M074301 

MO 

1-2 

N277903 

MF 

1-2 

Correct 


Response 

Sample 

Percent 

Cohen’s 

Codes 

Size 

Acreement 

Kanna 

2- 

1962 

98 

0.967 

2-3 

1956 

94 

0.877 

2-3 

1953 

98 

0.963 

2- 

1952 

99 

0.989 

2- 

1946 

99 

0.980 

2- 

1947 

95 

0.917 

4- 

1948 

100 

0.993 

2- 

1926 

99 

0.985 

2- 

1850 

99 

0.980 

4- 

1672 

90 

0.758 

2- 

1986 

99 

0.985 

3- 

1977 

100 

0.997 

3- 

6546^ 

99 

0.980 

2- 

1660 

95 

0.800 

2- 

1961 

98 

0.970 

2- 

1955 

100 

1.000 

2- 

1961 

98 

0.961 

3- 

1967 

99 

0.980 

4- 

1938 

98 

0.964 

4- 

1914 

99 

0.978 

3- 

1896 

100 

0.978 

5- 

1973 

100 

0.991 

4- 

1969 

98 

0.970 

5- 

1967 

99 

0.986 

5- 

1952 

100 

0.999 

4- 

1876 

100 

0.992 

3- 

1987 

97 

0.941 

2-3 

1971 

98 

0.966 

2- 

1960 

99 

0.974 

2-3 

1945 

99 

0.962 

4- 

1863 

97 

0.937 

3- 

1758 

98 

0.845 

2- 

1973 

100 

0.992 

2- 

1800 

100 

0.986 

' Cohen'S  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  by  key)  and  wrong. 

Rescored  responses  from  the  national  and  state  assessment  samples  contributed  to  these  statistics. 

3 M040001  was  erroneously  identified  as  appearing  only  in  the  national  assessment;  therefore,  the  item  was  rescored  at  a higher 
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Table  1-2 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Construeted-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  4' 


Item 

Block 

M041201 

MI 

M043501 

MM 

M066301 

ME 

M066501 

ME 

M066601 

ME' 

M066701 

ME 

M066801 

ME 

M066901 

ME 

M067901 

MG 

M068001 

MG 

M068002 

MG 

M068003 

MG 

M068004 

MG 

M068701 

ML 

M068901 

ML 

M069001 

ML 

M069101 

ML 

M072201 

MN 

M072202 

MN 

M072401 

MN 

M072501 

MN 

M072601 

MN 

M072701 

MN 

M074501 

MO 

M074701 

MO 

M074801 

MO 

M074901 

MO 

M075001 

MO 

M075101 

MO 

Range  of 
Response 
Codes 

Sample 

Size 

1-5 

1934 

1-5 

1882 

1-3 

1971 

1-3 

1949 

1-3 

1932 

1-3 

1883 

1-3 

1750 

i-5 

1756 

1-3 

1952 

1-3 

1944 

1-3 

1902 

1-3 

1679 

1-5 

1692 

1-3 

1968 

1-3 

1815 

1-3 

1724 

1-5 

1686 

1-3 

1952 

1-3 

1947 

1-3 

1848 

1-3 

1636 

1-3 

1360 

1-5 

1337 

1-3 

1969 

1-3 

1954 

1-3 

1930 

1-3 

1886 

1-3 

1770 

1-5 

1766 

Percent 

Intraclass 

Agreement 

Correlatioi 

84 

0.918 

91 

0.962 

98 

0.971 

98 

0.988 

92 

0.935 

96 

0.973 

96 

0.965 

92 

0.979 

98 

0.987 

99 

0.989 

98 

0.983 

98 

0.978 

94 

0.984 

99 

0.985 

98 

0.979 

91 

0.915 

94 

0.970 

99 

0.995 

97 

0.980 

94 

0.964 

93 

0.929 

98 

0.969 

94 

0.973 

92 

0.932 

98 

0.981 

99 

0.988 

95 

0.967 

99 

0.985 

90 

0.964 

' Rescored  responses  from  the  national  and  state  assessment  samples  contributed  to  these  statistics. 
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Table  1-3 

Score  Range,  Percent  Agreement,  and  Cohen’s  Kappa' 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Item 
Used  in  Main  Assessment  Scaling,  Grade  8^ 


Item 

Block 

Range  of 
Response 
Codes 

MO  19701 

MF 

1-2 

M019801 

MF 

1-3 

MO  19901 

MF 

1-3 

M020001 

MF 

1-2 

M020101 

MF 

1-2 

M020201 

MF 

1-2 

M020301 

MF 

1-4 

M020401 

MF 

1-2 

M020501 

MF 

1-2 

M020801 

MF 

1-6 

M020901 

MF 

1-2 

M021001 

MF 

1-2 

M021101 

MF 

1-3 

M021201 

MF 

1-3 

M021301 

MF 

1-2 

M021302 

MF 

1-2 

M046001 

MK 

1-5 

M046601 

MK 

1-4 

M046801 

MK 

1-5 

M046901 

MK 

1-5 

M047301 

MK 

1-4 

M047901 

MK 

1-3 

M050801 

MC 

1-2 

M050901 

MC 

1-4 

M051001 

MC 

1-2 

M051201 

MM 

1-2 

M051301 

MM 

1-2 

M051601 

MM 

1-2 

M052101 

MM 

1-3 

M052401 

MI 

1-2 

M052901 

MI 

1-2 

M053001 

MI 

1-2 

M061901 

MJ 

1-3 

M061902 

MJ 

1-3 

M061903 

MJ 

1-2 

M061904 

MJ 

1-3 

Correct 

Response 

Sample 

Percent 

Codes 

Size 

Agreement 

2- 

1742 

100 

2-3 

1786 

98 

2-3 

1781 

99 

2- 

1787 

100 

2- 

1784 

100 

2- 

1785 

97 

4- 

1784 

100 

2- 

1774 

99 

2- 

1774 

100 

6- 

1745 

99 

2- 

2188 

94 

2- 

1773  . 

100 

3- 

1741 

96 

3- 

1691 

98 

2- 

1704 

98 

2- 

1632 

98 

5- 

1785 

100 

4- 

1783 

98 

5- 

1784 

100 

5- 

1780 

100 

4- 

1775 

100 

3- 

1669 

100 

2- 

1773 

100 

4- 

1755 

100 

2- 

1708 

96 

2- 

1789 

100 

2- 

1778 

100 

2- 

1781 

99 

3- 

1758 

97 

2- 

1788 

96 

2- 

1766 

95 

2- 

1699 

94 

3- 

1796 

97 

2-3 

1788 

99 

2- 

1804 

99 

2-3 

1794 

99 

(continued) 
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Cohen’s 

Kappa 

0.987 

0.953 

0.983 

0.995 

0.990 

0.883 

0.998 

0.989 

0.991 

0.976 

0.897 

0.994 

0.920 

0.968 

0.968 

0.951 

0.983 

0.957 

0.988 

0.988 

0.982 

0.991 

0.995 

0.993 

0.939 

0.992 

0.986 

0.974 

0.944 

0.929 

0.898 

0.897 

0.916 

0.952 

0.925 

0.982. 


Table  1-3  (continued) 

Score  Range,  Percent  Agreement,  and  Cohen ’s  Kappa 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  8 


Item 

Block 

Range  of 
Response 
Codes 

Correct 

Response 

Codes 

Sample 

Size 

Percent 

A&reement 

Cohen’s 

Kanna 

M061905 

MJ 

1-4 

4- 

1602 

95 

0.896 

M061907 

MJ 

1-3 

3- 

1793 

96 

0.906 

M061908 

MJ 

1-3 

3- 

1720 

98 

0.904 

' Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  as  key)  and  wrong. 

^ Rescored  responses  from  the  national  and  state  assessment  samples  contributed  to  these  statistics. 


Table  1-4 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  8 


Range  of 


Item 

Block 

Response 

Codes 

M051101 

MC 

1-5 

M052201 

MM 

1-5 

M053101 

MI 

1-5 

M066301 

ME 

1-3 

M066501 

ME 

1-3 

M066601 

ME 

1-3 

M067201 

ME 

1-3 

M067501 

ME 

1-5 

M067901 

MG 

■ 1-3 

M068003 

MG 

1-3 

M068005 

MG 

1-3 

M068006 

MG 

1-3 

M068008 

MG 

1-3 

M068201 

MG 

1-5 

M069301 

ML 

1-3 

M069601 

ML 

1-3 

M069701 

ML 

1-3 

M069901 

ML 

1-3 

Sample 

Percent 

Intraclass 

Size 

Agreement 

Correlatioi 

1718 

85 

0.903 

1759 

93 

0.968 

1698 

90 

0.937 

1789 

99 

0.972 

2349 

96 

0.976 

1770 

94 

0.958 

1762 

91 

0.944 

1750 

94 

0.942 

1785 

98 

0.991 

1779 

99 

0.988 

1785 

98 

0.981 

1741 

94 

0.950 

1462 

92 

0.898 

1001 

88 

0.939 

1818 

99 

0.992 

1794 

95 

0.940 

1769 

100 

0.997 

1514 

95 

0.953 

(continued) 
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Table  1-4  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  8' 


Range  of 

Item 

Block 

Response 

Codes 

Sample 

Size 

Percent 

Aereement 

Intraclass 

Correlation 

M070001 

M072901 

M073401 

M073501 

M073601 

M075301 

M075401 

M075601 

M075801 

M076001 

ML 

MN 

MN 

MN 

MN 

MO 

MO 

MO 

MO 

MO 

1-4 

1-3 

1-3 

1-3 

1-5 

1-3 

1-3 

1-3 

1-3 

1-5 

1528 

1789 

1752 

1622 

1247 

1774 

1774 

1837 

1671 

1506 

90 

93 

97 

98 
87 
98 

94 

95 

90 

91 

0.955 

0.951 

0.963 

0.978 

0.948 

0.980 

0.942 

0.953 

0.939 

0.947 

Rescored  responses  from  the  national  and  state  assessment  samples  contributed  to  these  statistics. 


Table  1-5 

Score  Range,  Perc^  Agreement,  and  Cohen’s  Kappa' 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 

Used  in  Main  Assessment  Scaling,  Grade  12 


Item 

Block 

Range  of 
Response 
Codes 

MO  19701 

MF 

1-2 

MO  19801 

MF 

1-3 

MO 19901 

MF 

1-3 

M020001 

MF 

1-2 

M020101 

MF 

1-2 

M020201 

MF 

1-2 

M020301 

MF 

1-4 

M020401 

MF 

1-2 

M020501 

MF 

1-2 

M020801 

MF 

1-6 

M020901 

MF 

1-2 

M021001 

MF 

1-2 

M021101 

MF 

1-3 

M021201 

MF 

1-3 

M021301 

MF 

1-2 

M021302 

MF 

1-2 

ERIC 


Correct 


Response 

Sample 

Percent 

Cohen’s 

Codes 

Size 

Aereement 

Kanna 

2- 

163 

99 

0.978 

2-3 

168 

98 

0.957 

2-3 

166 

99 

0.980 

2- 

167 

100 

1.000 

2- 

167 

99 

0.973 

2- 

168 

96 

0.890 

4- 

170 

100 

1.000 

2- 

167 

100 

1.000 

2- 

161 

99 

0.988 

6- 

165 

99 

0.986 

2- 

198 

93 

0.888 

2- 

160 

99 

0.989 

3- 

154 

97 

0.940 

3- 

158 

98 

0.964 

2- 

155 

97 

0.949 

2- 

143 

99 

0.984 

(continued) 
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Table  1-5  (continued) 

Score  Range,  Percent  Agreernent,  and  Cohen ’s  Kappa 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Item 

Block 

Range  of 
Response 
Codes 

M046001 

MK 

1-5 

M046601 

MK 

1-4 

M046801 

MK 

1-5 

M046901 

MK 

1-5 

M047301 

MK 

1-4 

M047901 

MK 

1-3 

M050801 

MC 

1-2 

M050901 

MC 

1-4 

M051001 

MC 

1-2 

M051201 

MM 

1-2 

M051301 

MM 

1-2 

M051601 

MM 

1-2  , 

M052101 

MM 

1-3 

M052401 

MI 

1-2 

M052901 

MI 

1-2 

M053001 

MI 

1-2 

M061901 

Ml  . 

1-3 

M061902 

Ml 

1-3 

M061903 

Ml 

1-2 

M061904 

Ml 

1-3 

M061905, 

MJ. 

1-4 

M061907 

Ml 

1-3 

M061908 

MJ 

1-3 

Correct 

Response 

Sample 

Percent 

Codes 

Size 

Asreement 

5- 

168 

99 

4- 

165 

96 

5- 

165 

100 

5- 

165 

100 

4- 

163 

99 

3- 

147 

99 

2- 

160 

100 

4- 

156 

100 

2- 

154 

82 

2- 

163 

100 

2- 

163 

99 

2- 

159 

99 

3- 

159 

98 

2- 

164 

98 

2- 

157 

94 

2- 

149 

97 

3- 

185 

98 

2-3 

179 

98 

2- 

181 

98 

.2-3 

183 

99 

4- 

153 

92 

3- 

171 

97 

3- 

175 

99 

Cohen’s 

Kappa 

0.979 

0.906 

1.000 

1.000 

0.969 

0.976 

1.000 

1.000 

0.671 

1.000 

0.967 

0.971 

0.964 

0.959 

0.892 

0.942 

0.954 

0.932 

0.920 

0.990 

0.841 

0.935 

0.958 


' Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  as  key)  and  wrong. 
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Item 

M051101 

M052201 

M053101 

M066301 

M066501 

M066601 

M067201 

M067501 

M067901 

M068003 

M068005 

M068006 

M068008 

M068201 

M069301 

M069601 

M069701 

M069901 

M070001 

M072901 

M073401 

M073501 

M073601 

M075301 

M075401 

M075601 

M075801 

M076001 
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Table  1-6 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Range  of 


Block 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Intraclass 

Correlation 

MC 

1-5 

152 

80 

0.886 

MM 

1-5 

162 

95 

0.983 

MI 

1-5 

149 

89 

0.913 

ME 

1-3 

158 

99 

0.995 

ME 

1-3 

728 

96 

0.967 

ME 

1-3 

154 

97 

0.979 

ME 

1-3 

159 

90 

0.946 

ME 

1-5 

160 

92 

0.938 

MG 

1-3 

170 

99 

0.995 

MG 

1-3 

163 

99 

0.989 

MG 

1-3 

160 

98 

0.990 

MG 

1-3 

156 

92 

0.933 

MG 

1-3 

128 

93 

0.890 

MG 

1-5 

85 

88 

0.943 

ML 

1-3 

185 

99 

0.993 

ML 

1-3 

180 

93 

0.908 

ML 

1-3 

176 

99 

0.997 

ML 

1-3 

148 

95 

0.954 

ML 

1-4 

147 

90 

0.967 

MN 

1-3 

161 

94 

0.955 

MN 

1-3 

159 

96 

0.955 

MN 

1-3 

147 

100 

1.000 

MN 

1-5 

no 

89 

0.963 

MO 

1-3 

165 

100 

1.000 

MO 

1-3 

161 

96 

0.966 

MO 

1-3 

166 

94 

0.949 

MO 

1-3 

151 

93 

0.959 

MO 

1-5 

128 

91 

0.958 
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Table  1-7 

Score  Range,  Percent  Agreement,  and  Cohen*  s Kappa^ 
for  the  Dichotomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  4 


Item 

Block 

Range  of 
Response 
Codes 

Correct 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Cohen’s 

Kanna 

K031004 

SC 

1-2 

2- 

678 

95 

0.872 

K031101 

SD 

1-2 

2- 

675 

97 

0.914 

K031102 

SD 

1-2 

2- 

671 

95 

0.887 

K031103 

SD 

1-2 

2- 

658 

95 

0.896 

K031104 

SD 

1-2 

2- 

652 

98 

0.930 

K031508 

SH 

1-2 

2- 

415 

95 

0.862 

K033001 

SK 

1-2 

2- 

469 

95 

0.865 

^ Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  by  key)  and  wrong.  These  items  are  dichotomized  into  right  (as  indicated  by  key)  and  wrong. 


Table  1-8 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  4 


Item 

Block 

Range  of 
Response 
Codes 

K031001 

SC 

1-3 

K031002 

SC 

1-3 

K031003 

sc 

1-3 

K031005 

sc 

1-3 

K031006 

sc 

1-3 

K031007 

sc 

1-3 

K031105 

SD 

1-3 

K031107 

SD 

1-4 

K031201 

SE 

1-3 

K031202 

SE 

1-3 

K031203 

SE 

1-4 

K031204 

SE 

1-5 

K031210 

SE 

1-3 

K031211 

SE 

1-3 

K031301 

SF 

1-4 

K031302 

SF 

1-3 

K031303 

SF 

1-3 

Sample 

Percent 

Intraclass 

Size 

Agreement 

Correlation 

759 

96 

0.927 

748 

88 

0.872 

732 

91 

0.837 

655 

87 

0.846 

579 

93 

0.951 

499 

84 

0.870 

633 

99 

0.978 

543 

91 

0.949 

700 

99 

0.988 

698 

94 

0.869 

687 

96 

0.990 

687 

89 

0.936 

481 

87 

0.886 

382 

88 

0.854 

656 

94 

0.963 

552 

94 

0.854 

488 

93 

0.926 

(continued) 
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Table  1-8  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  4 


Item 

Block 

Range  of 
Response 
Codes 

K031304 

SF 

1-3 

K031309 

SF 

1-4 

K031401 

SG 

1-4 

K031402 

SG 

1-3 

K031403 

SG 

1-3 

K031404 

SG 

1-3 

K031407 

SG 

1-3 

K031408 

SG 

1-3 

K031409 

SG 

1-3 

K03I410 

SG 

1-3 

K031501 

SH 

1-3 

K031502 

SH 

1-3 

K031503 

SH 

1-3 

K031505 

SH 

1-3 

K031506 

SH 

1-5 

K031507 

SH 

1-3 

K031509 

SH 

1-3 

K031602 

SI 

1-3 

K031603 

SI 

1-3 

K031604 

SI 

1-3 

K031606 

SI 

1-3 

K031607 

SI 

1-4 

K031608 

SI 

1-3 

K031609 

SI 

1-3 

K031901 

SJ 

1-3 

K032001 

SJ 

1-3 

K032501 

SJ 

1-3 

K032502 

SJ 

1-3 

K032601 

SJ 

1-3 

K032602 

SJ 

1-3 

K033101 

SK 

1-3 

K033501 

SK 

1-3 

K033502 

SK 

1-3 

K033503 

SK 

1-3 

K034001 

SL 

1-3 

K034101 

SL 

1-3 

KW34101 

SL 

1-3 

KX34101 

SL 

1-3 

KZ34101 

SL 

1-3 

Sample 

Percent 

Intraclass 

Size 

Agreement 

Correlation 

438 

94 

0.940 

617 

93 

0.957 

604 

88 

0.902 

611 

93 

0.955 

604 

94 

0.934 

598 

95 

0.976 

580 

91 

0.870 

561 

98 

0.989 

550 

95 

0.948 

536 

96 

0.945 

631 

98 

0.984 

622 

94 

0.880 

635 

94 

0.949 

622 

96 

0.970 

610 

88 

0.938 

482 

94 

0.915 

633 

94 

0.944 

603 

98 

0.987 

593 

98 

0.984 

594 

99 

0.991 

584 

96 

0.940 

565 

93 

0.960 

518 

93 

0.895 

480 

96 

0.920 

463 

89 

0.929 

464 

97 

0.981 

448 

95 

0.962 

440 

96 

0.969 

422 

89 

0.866 

390 

91 

0.915 

469 

93 

0.832 

448 

93 

0.894 

422 

90 

0.920 

398 

89 

0.877 

459 

91 

0.847 

466 

89 

0.906 

464 

89 

0.936 

460 

95 

0.942 

456 

86 

0.868 

(continued) 
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Table  1-8  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  4 


Range  of 


Response 

Sample 

Percent 

Intraclass 

Item 

Block 

Codes 

Size 

Agreement 

Correlation 

K034802 

SM 

1-3 

509 

94 

0.927 

K034901 

SM 

1-3 

503 

96 

0.932 

K034902 

SM 

1-3 

508 

95 

0.933 

K035201 

SM 

1-3 

477 

92 

0.939 

K035301 

SM 

1-4 

455 

94 

0.940 

K035601 

SN 

1-3 

612 

95 

0.936 

K035801 

SN 

1-3 

626 

93 

0.954 

K035901 

SN 

1-3 

605 

97 

0.936 

K036101 

SN 

1-3 

586 

94 

0.952 

K036301 

SN 

1-3 

474 

97 

0.961 

K037301 

SO 

1-3 

452 

97 

0.937 

K037401 

so 

1-3 

437 

95 

0.922 

K037501 

so. 

1-3 

441 

97 

0.884 

K037601 

so 

1-3 

433 

93 

0.909 

K037701 

so 

1-4 

419 

97 

0.982 

K037702 

so 

1-3 

359 

93 

0.915 

K038801 

ST 

1-4 

510 

97 

0.967 

K038901 

ST 

1-4 

509 

91 

0.923 

K039201 

ST 

1-3 

464 

96 

0.937 

K039301 

ST 

1-3 

452 

94 

0.955 

K039401 

ST 

1-4 

433 

94 

0.916 

K039801 

SU 

1-3 

454 

96 

0.961 

K039901 

SU 

1-5 

460 

91 

0.963 

K040001 

SU 

1-3 

440 

90 

0.935 

K040301 

SU 

1-3 

382 

89 

0.879 

K040401 

SU 

1-4 

342 

95 

0.950 

K040501 

SU 

1-3 

281 

97 

0.977 

795 


Table  1-9 

Score  Range,  Percent  Agreement,  and  Cohen’s  Kappa' 
for  the  Dichotomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  8^ 


Range  of 

Correct 

■ 

Response 

Response 

Sample 

Percent 

Cohen’s 

Item 

Block 

Codes 

Codes 

Size 

Agreement 

Kanna 

K038101 

SO 

1-2 

2- 

1230 

98 

0.928 

K040806 

SE 

1-2 

2- 

792 

91 

0.750 

K040904 

SG 

1-2 

2- 

1669 

98 

0.956 

K041101 

SG 

1-2 

2- 

1654 

98 

0.959 

K042603 

SJ 

1-2 

2- 

1158 

95 

0.850 

K048103 

SL 

1-2 

2- 

1077 

94 

0.869 

' Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  by  key)  and  wrong. 

^ Rescored  responses  from  the  national  and  state  assessment  samples  contributed  to  these  statistics. 


Table  MO 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  S' 


Item 

Block 

Range  of 
Response 
Codes 

K031301 

SF 

1-4 

K031302 

SF 

1-3 

K031305 

SF 

1-3 

K031306 

SF 

1-3 

K031307 

SF 

1-3 

K031308 

SF 

1-3 

K031309 

SF 

1-4 

K031602 

SI 

1-3 

K031603 

SI 

1-3 

K031604 

SI 

1-3 

K031606 

SI 

1-3 

K031607 

SI 

1-4 

K031608 

SI 

1-3 

K031609 

SI 

1-3 

K031610 

SI 

1-3 

K031611 

SI 

1-3 

K031613 

SI 

1-3 

Sample 

Percent 

Intraclass 

Size 

Agreement 

Correlation 

1901 

98 

0.918 

1876 

89 

0.797 . 

1828 

94 

0.877 

1809 

98 

0.867 

1693 

94 

0.938 

1496 

97 

0.921 

1898 

97 

0.925 

1684 

98 

0.983 

1676 

99 

0.961 

1680 

100 

0.994 

1676 

95 

0.945 

1675 

90 

0.927 

1668 

92 

0.891 

1667 

96 

0.958 

1675 

98 

0.979 

1658 

97 

0.951 

1602 

99 

0.961 

(continued) 
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Table  I- 10  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed^Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  5* 


Range  of 
Response 

Item 

Block 

Codes 

K035601 

SN 

1-3 

K035801 

SN 

1-3 

K035901 

SN 

1-3 

K036101 

SN 

1-3 

K036301 

SN 

1-3 

K036401 

SN 

1-3 

K036402 

SN 

1-3 

K036403 

. SN 

1-3 

K036404 

SN 

1-3 

K036701 

SN 

1-3 

K036801 

SN 

1-3 

K037301 

SO 

1-3 

K037401 

so 

1-3 

K037501 

so 

1-3 

K037601 

so 

1-3 

K037701 

so 

1-4 

K037703 

so 

1-3 

K038201 

so 

1-3 

K038301 

so 

1-5 

K040601 

sc 

1-3 

K040603 

sc 

1-3 

K040604 

sc 

1-3 

K040605 

sc 

1-3 

K040606 

sc 

1-4 

K040607 

sc 

1-4 

K040608 

sc 

1-4 

K040609 

sc 

1-4 

K040610 

sc 

1-4 

K040701 

SD 

1-4 

K040702 

SD 

1-4 

K040704 

SD 

1-3 

K040705 

SD 

1-3 

K040708 

SD 

1-3 

K040711 

SD 

1-3 

K040713 

SD 

1-4 

K040802 

SE 

1-3 

K040803 

SE 

1-3 

K040804 

SE 

1-3 

Sample 

Percent 

Intraclass 

Size 

Aereement 

Correlation 

1685 

93 

0.936 

1688 

94 

0.955 

1683 

95 

0.961 

1683 

94 

0.942 

1677 

95 

0.974 

1670 

97 

0.955 

1672 

93 

0.930 

1669 

92 

0.876 

1666 

92 

0.796 

1671 

97 

0.954 

1656 

97 

0.965 

1271 

93 

0.947 

1271 

93 

0.938 

1263 

96 

0.916 

1269 

89 

0.926 

1269 

99 

0.985 

1266 

91 

0.900 

1225 

92 

0.849 

1132 

85 

0.899 

2051 

99 

0.988 

1832 

95 

0.956 

1778 

93 

0.916 

1665 

95 

0.970 

1544 

94 

0.959 

2027 

94 

0.974 

2005 

91 

0.959 

1978 

95 

0.980 

1949 

93 

0.967 

1892 

93 

0.942 

1889 

94 

0.955 

1883 

95 

0.958 

1872 

93 

0.864 

1855 

92 

0.911 

1768 

92 

0.955 

1561 

84 

0.898 

1775 

98 

0.980 

1522 

96 

0.928 

1117 

94 

0.903 

(continued) 
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Table  I-IO  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade 


Item 

Block 

Range  of 
Response 
Codes 

K040805 

SE 

1-3 

K040808 

SE 

1-3 

K040901 

SG 

1-3 

K040902 

SG 

1-3 

K040903 

SG 

1-3 

K040905 

SG 

1-3 

K041002 

SG 

1-3 

K041004 

SG 

1-3 

K041201 

SG 

1-3 

K041202 

SG 

1-3 

K041306 

SH 

1-3 

K041307 

SH 

1-3 

K041401 

SH 

1-3 

K041402 

SH 

1-3 

K041403 

SH 

1-3 

K041901 

SJ 

1-3 

K042001 

SJ 

1-3 

K042101 

SJ 

1-3 

K042102 

SJ 

1-3 

K042201 

SJ 

1-3 

K042601 

SJ 

1-3 

K042602 

SJ 

1-4 

K043001 

SK 

1-3 

K043101 

SK 

1-3 

K043102 

SK 

1-4 

K043103 

SK 

1-3 

K043501 

SK 

1-3 

K043601 

SK 

1-3 

K043602 

SK 

1-3 

K043603 

SK 

1-3 

K044101 

ST 

1-3 

K044201 

ST 

1-3 

K044301 

ST 

1-3 

K044401 

ST 

1-3 

K044901 

ST 

1-3 

K045001 

ST 

1-3 

K045101 

ST 

1-4 

K045102 

ST 

1-4 

Sample 

Percent 

Intraclass 

Size 

Agreement 

Correlation 

1030 

95 

0.922 

1673 

98 

0.989 

1670 

96 

0.981 

1672 

93 

0.946 

1669 

91 

0.938 

1668 

92 

0.915 

1665 

86 

0.910 

1654 

96 

0.983 

1650 

91 

0.911 

1508 

90 

0.811 

1678 

87 

0.903 

1685 

91 

0.894 

1668 

96 

0.958 

1661 

99 

0.988 

1643 

94 

0.890 

1254 

95 

0.927 

1259 

91 

0.888 

1256 

98 

0.959 

1258 

97 

0.974 

1259 

97 

0.915 

1215 

89 

0.875 

1190 

94 

0.956 

1259 

95 

0.919 

1260 

89 

0.888 

1261 

85 

0.934 

1254 

92 

0.818 

1246 

94 

0.936 

1178 

89 

0.881 

1119 

95 

0.877 

1110 

95 

0.916 

1268 

98 

0.974 

1270 

89 

0.904 

1270 

95 

0.965 

1268 

85 

0.860 

1256 

93 

0.916 

1237 

90 

0.898 

1155 

88 

0.928 

1092 

89 

0.907 

(continued) 
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Table  I-IO  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  8^ 


Item 

Block 

Range  of 
Response 
Codes 

K045301 

SU 

1-3 

K045601 

su 

1-4 

KD45701 

SU 

1-3 

K045801 

su 

1-3 

K046301 

su 

1-3 

K046401 

su 

1-3 

K046501 

su 

1-3 

K046601 

su 

1-3 

K046701 

su 

1-4 

K047201 

SL 

1-4 

K047301 

SL 

1-3 

K047401 

SL 

1-3 

K047901 

SL 

1-3 

K048001 

SL 

1-3 

K048101 

SL 

1-4 

K048102 

SL 

1-3 

K048601 

SM 

1-3 

K048901 

SM 

1-3 

K049001 

SM 

1-3 

K049301 

SM 

1-3 

K049401 

SM 

1-3 

K049402 

SM 

1-3 

K049403 

SM 

1-3 

K049404 

SM 

1-4 

Sample 

Percent 

Intraclass 

Size 

Aereement 

Correlation 

1252 

93 

0.951 

1257 

96 

0.954 

1259 

90 

0.921 

1251 

94 

0.933 

1246 

93 

0.879 

1218 

94 

0.905 

1215 

95 

0.954 

1192 

91 

0.942 

1137 

88 

0.873 

1260 

88 

0.948 

1262 

96 

0.969 

1254 

92 

0.942 

1237 

92 

0.952 

1209 

99 

0.986 

1156 

90 

0.896 

1093 

95 

0.866 

1272 

93 

0.931 

1278 

98 

0.985 

1278 

100 

0.981 

1254 

98 

0.941 

1234 

94 

0.928 

1236 

89 

0.919 

1190 

88 

0.875 

1156 

84 

0.833 

‘ Rescored  responses  from  the  national  and  state  assessment  samples  contributed  to  these  statistics. 


Table  Ml 

Score  Range,  Percent  Agreement,  and  Cohen* s Kappa} 
for  the  Dichotomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Item 

Block 

Range  of 
Response 
Codes 

Correct 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Cohen’s 

Kanna 

K040806 

SE 

1-2 

2- 

401 

86 

0.705 

K048103 

SL 

1-2 

2- 

361 

94 

0.884 

K049708 

SF 

1-2 

2- 

690 

98 

0.951 

K051002 

SJ 

1-2 

2- 

450 

92 

0.854 

K051003 

SJ 

1-2 

2- 

448 

96 

0.911 

K051004 

SJ 

1-2 

2- 

444 

96 

0.915 

K051701 

SK 

1-2 

2- 

455 

96 

0.918 

K054004 

SO 

1-2 

2- 

431 

97 

0.902 

^ Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  by  key)  and  wrong. 


Table  1-12 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed- Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Range  of 


Item 

Block 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Intraclass 

Correlation 

K040802 

SE 

1-3 

660 

99 

0.987 

K040803 

SE 

1-3 

603 

97 

0.946 

K040804 

SE 

1-3 

502 

93 

0.953 

K040805 

SE 

1-3 

471 

96 

0.959 

K040808 

SE 

1-3 

641 

98 

0.991 

K041306 

SH 

1-3 

628 

88 

0.906 

K041307 

SH 

1-3 

624 

91 

0.921 

K041401 

SH 

1-3 

630 

93 

0.924 

K041402 

SH 

1-3 

626 

99 

0.990 

K041403 

SH 

1-3 

614 

94 

0.897 

K041404 

SH 

1-3 

608 

95 

0.963 

K041406 

SH 

1-3 

569 

96 

0.898 

K047201 

SL 

1-4 

459 

90 

0.959 

K047301 

SL 

1-3 

458 

96 

0.971 

K047401 

SL 

1-3 

458 

92 

0.954 

K047901 

SL 

1-3 

447 

94 

0.962 

K048001 

SL 

1-3 

437 

98 

0.972 

(continued) 
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Table  1*12  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Range  of 


Item 

Block 

Response 

Codes 

• Sample  ' 
Size 

Percent 

A2reement 

Intraclass 

Correlation 

K048101 

SL 

1-4 

404 

84 

0.861 

K048102 

SL 

1-3 

364 

93 

0.850 

K048601 

SM 

1-3 

476 

92 

0.937 

K048901 

SM 

1-3 

476 

98 

0.985 

K049001 

SM 

1-3 

476 

99 

0.940 

K049301 

SM 

1-3 

475 

96 

0.921 

K049401 

SM 

1-3 

463 

94 

0.938 

K049402 

SM 

1-3 

457 

86 

0.894 

K049403 

SM 

1-3 

432 

87 

0.871 

K049404 

SM 

1-4 

418 

81 

0.827 

K049501 

SC 

1-4 

752 

98 

0.970 

K049502 

sc 

1-5 

743 

92 

0.930 

K049503 

sc 

1-3 

721 

94 

0.954 

K049504 

sc 

1-3 

699 

93 

0.946 

K049505 

sc 

1-3 

667 

93 

0.945 

K049506 

sc 

1-4 

597 

96 

0.917 

K049601 

SD 

1-4 

701 

88 

0.894 

K049602 

SD 

1-5 

697 

96 

0.983 

K049603 

SD 

1-5 

643 

82 

0.883 

K049604 

SD 

1-3 

510 

94 

0.949 

K049701 

SF 

1-3 

692 

99 

0.968 

K049702 

SF 

1-3 

692 

98 

0.973 

K049703 

SF 

1-3 

681 

93 

0.940  ' 

K049704 

SF 

1-3 

670 

95 

0.951 

K049705 

SF 

1-4 

615 

88 

0.954 

K049706 

SF 

1-3 

586 

83 

0.874 

K049707 

SF 

1-5 

509 

84 

0.938 

K049802 

SG 

1-3 

608 

95 

0.971 

K049804 

SG 

1-3 

609 

95 

0.954 

K049807 

SG 

1-4 

610 

90 

0.941 

K049809 

SG 

1-3 

599 

88 

0.880 

K049810 

SG 

1-4 

592 

90 

0.960 

K049812 

SG 

1-3 

588 

95 

0.940 

K049813 

SG 

1-3 

564 

92 

0.929 

K049814 

SG 

1-3 

550 

94 

0.932 

K049815 

SG 

1-4 

472 

91 

0.921 

K049901 

SI 

1-3 

604 

94 

0.963 

K049902 

SI 

1-3 

601 

87 

0.910 

K049903 

SI 

1-3 

599 

90 

0.931 

K049904 

SI 

1-3 

600 

93 

0.941 

(continued) 
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Table  1-12  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Range  of 


Item 

Block 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Intraclass 

Correlation 

K049907 

SI 

1-3 

593 

85 

0.916 

K049908 

SI 

1-3 

585 

97 

0.980 

K049909 

SI 

1-4 

584 

94 

0.952 

K049911 

SI 

1-3 

560 

94 

0.955 

K049912 

SI 

1-5 

534 

81 

0.906 

K049914 

SI 

1-3 

544 

93 

0.931 

K050401 

SJ 

1-3 

457 

93 

0.898 

K050501 

SJ 

1-4 

453 

91 

0.962 

K050901 

SJ 

1-3 

454 

98 

0.984 

K051001 

SJ 

1-3 

453 

93 

0.976 

K051101 

SJ 

1-3 

434 

96 

0.948 

K051102 

SJ 

1-3 

417 

93 

0.852 

K051201 

SJ 

1-3 

380 

92 

0.878 

K051801 

SK 

1-3 

453 

91 

0.927 

K052301 

SK 

1-3 

448 

98 

0.983 

K052401 

SK 

1-4 

450 

90 

0.959 

K052402 

SK 

1-3 

441 

97 

0.975 

K052501 

SK 

1-4 

441 

90 

0.959 

K052502 

SK 

1-4 

435 

91 

0.926 

K052503 

SK 

1-3 

422 

94 

0.885 

K052901 

SN 

1-4 

641 

89 

0.952 

K053001 

SN 

1-3 

638 

92 

0.914 

K053101 

SN 

1-3 

641 

99 

0.993 

K053102 

SN 

1-3 

636 

92 

0.944 

K053601 

SN 

- 1-5 

621 

91 

0.944 

K053701 

SN 

1-3 

594 

95 

0.952 

K053801 

SN 

1-3 

582 

90 

0.900 

K053901 

SN 

1-3 

540 

94 

0.895 

K054001 

SO 

1-4 

447 

98 

0.977 

K054002 

so 

1-3 

444 

93 

0.915 

K054003 

so 

1-3 

436 

99 

0.985 

K054005 

so 

1-3 

422 

96 

0.953 

K054006 

so 

1-3 

416 

97 

0.969 

K054007 

so 

1-3 

396 

85 

0.793 

K054008 

so 

1-4 

342 

79 

0.831 

(continued) 
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Table  1-12  (continued) 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Science  Items 
Used  in  Main  Assessment  Scaling,  Grade  12 


Range  of 

Response  Sample  Percent  Intraclass 

Item  Block  Codes  Size  Agreement  Correlation 


K057401 

ST 

1-3 

465 

92 

0.943 

K057501 

ST 

1-3 

468 

91 

0.909 

K057601 

ST 

1-3 

469 

97 

0.970 

K057701 

ST 

1-3 

470 

98 

0.973 

K058001 

ST 

1-3 

469 

95 

0.896 

K058201 

ST 

1-4 

446 

88 

0.925 

K058301 

ST 

1-4 

423 

86 

0.906 

K058401 

ST 

1-3 

394 

93 

0.963 

K058501 

ST 

1-4 

364 

91 

0.929 

K059001 

SU 

1-3 

475 

89 

0.873 

K059101 

SU 

1-3 

470 

95 

0.941 

K059201 

SU 

1-4 

474 

93 

0.974 

K059301 

SU 

1-4 

473 

99 

0.985 

K059801 

SU 

1-3 

448 

91 

0.920 

K059901 

SU 

1-4 

444 

95 

0.964 

K060001 

SU 

1-3 

433 

92 

0.954 

K060101 

SU 

1-4 

423 

89 

0.895 

K089811 

SG 

1-3 

587 

92 

0.911 

Table  1-13 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Reading  Items 

Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  9 

Range  of 
Response 

Sample 

Percent 

Intraclass 

Item 

Block 

Codes 

Size 

Agreement 

Correlation 

N001507 

BH 

1-6 

252 

92 

0.947 

N002804 

BL 

1-5 

184 

95 

0.958 

N003104 

BM 

1-5 

177 

87 

0.879 

N003704 

BN 

1-4 

192 

92 

0.902 

N008905 

BJ 

1-6 

235 

94 

0.972 

803 


Table  M4 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Reading  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  13 


Item 

Block 

Range  of 
Response 
Codes 

Sample 

Size 

Percent 

Agreement 

Intraclass 

Correlation 

N001507 

BH 

1-6 

285 

89 

0.901 

N001904 

BJ 

1-5 

256 

87 

0.914 

N002302 

BK 

1-9 

0 

0 

0.000 

N002804 

BL 

1-5 

309 

90 

0.911 

N003104 

BM 

1-5 

272 

90 

0.909 

N003704 

BN 

1-4 

236 

89 

0.889 

N004303 

BO 

1-4 

259 

91 

0.908 

N004605 

BP 

1-5 

277 

94 

0.969 

Table  M5 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Reading  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  17 

Range  of 


Item 

Block 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Intraclass 

Correlation 

N001507 

BH 

1-6 

244 

90 

0.929 

N001904 

BJ 

1-5 

261 

91 

0.931 

N002302 

BK 

1-9 

0 

0 

0.000 

N002804 

BL 

1-5 

262 

90 

0.911 

N003104 

BM 

1-5 

239 

89 

0.908 

N003704 

BN 

1-4 

219 

89 

0.908 

N004303 

BO 

1-4 

180 

83 

0.879 

N004605 

BP 

1-5 

230 

95 

0.967 

N015905 

BQ 

1-4 

201 

89 

0.932 

O )4 


Table  M6 

Score  Range,  Percent  Agreement,  and  Cohen*  s Kappa} 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  9 


Item 

Block 

Range  of 
Response 
Codes 

Correct 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Cohen’s 

Kanna 

N270001 

Ml 

1-2 

1- 

618 

99 

0.972 

N270901 

Ml 

1-2 

1- 

618 

100 

0.973 

N271101 

M2 

1-2 

1- 

614 

98 

0.955 

N275401 

M2 

1-2 

1- 

614 

98 

0.951 

N276001 

M2 

1-2 

1- 

614 

99 

0.976 

N276002 

M2 

1-2 

1- 

614 

98 

0.963 

N276021 

M3 

1-2 

1- 

575 

99 

0.979 

N276022 

M3 

1-2 

1- 

575 

99 

0.979 

N276101 

Ml 

1-2 

1- 

618 

100 

0.992 

N276801 

Ml 

1-2 

1- 

618 

100 

0.961 

N276802 

Ml 

1-2 

1- 

618 

100 

0.979 

N276803 

Ml 

1-2 

1- 

618 

98 

0.963 

N276821 

M3 

1-2 

1- 

575 

100 

0.978 

N276822 

M3 

1-2 

1- 

575 

99 

0.962 

N276823 

M3 

1-2 

1- 

575 

99 

0.981 

N277501 

M2 

1-2 

1- 

614 

99 

0.967 

N277601 

M2 

1-4 

1- 

614 

99 

0.979 

N277602 

M2 

1-5 

1- 

614 

99 

0.987 

N277603 

M2 

1-2 

1- 

614 

99 

0.971 

N277621 

M3 

1-2 

1- 

575 

99 

0.971 

N277622 

M3 

1-2 

1- 

575 

99 

0.959 

N277623 

M3 

1-2 

1- 

575 

99 

0.949 

N284001 

Ml 

1-2 

1- 

618 

99 

0.981 

N284002 

Ml 

1-2 

1- 

618 

99 

0.957 

N284021 

M3 

1-2 

1- 

575 

99 

0.960 

N284022 

M3 

1-2 

1- 

575 

99 

0.959 

N286101 

Ml 

1-2 

1- 

618 

99 

0.980 

N286102 

M2 

1-2 

1- 

614 

98 

0.966 

* Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  as  key)  and  wrong. 
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Table  M7 

Score  Range,  Percent  Agreement,  and  Cohen* s Kappa} 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  13 


Item 

Block 

Range  of 
Response 
Codes 

Correct 

Response 

Codes 

Sample 

Size 

Percent 

Aereement 

Cohen’s 

Kanna 

N256101 

M2 

1-2 

1- 

621 

99 

0.974 

N257601 

Ml 

1-2 

1- 

643 

99 

0.976 

N263101 

Ml 

1-2 

1- 

643 

99 

0.975 

N264521 

M3 

1-2 

1- 

624 

98 

0.968 

N269201 

M2 

1-2 

1- 

621 

96 

0.857 

N275001 

Ml 

1-2 

1- 

643 

99 

0.981 

N276801 

Ml 

1-2 

1- 

643 

100 

0.946 

N276802 

Ml 

1-2 

1- 

643 

100 

0.928 

N276803 

Ml 

1-2 

1- 

643 

98 

0.951 

N276821 

M3 

1-2 

1- 

624 

100 

1.000 

N276822 

M3 

1-2 

1- 

624 

100 

0.901 

N276823 

M3  , 

1-2 

1- 

624 

100 

0.979 

N277601 

Ml 

1-4 

1- 

643 

99 

0.935 

N277602 

Ml 

1-5 

1- 

643 

99 

0.981 

N277603 

Ml 

1-2 

1- 

643 

99 

0.964 

N277901 

M2 

1-2 

1- 

621 

100 

0.950 

N277902 

M2 

1-2 

1- 

621 

100 

0.907 

N277903 

M2 

1-2 

2- 

621 

100 

0.980 

N280621 

M3 

1-2 

1- 

624 

99 

0.976 

N280622 

M3 

1-2 

1- 

624 

98 

0.968 

N280623 

M3 

1-2 

1- 

624 

99 

0.978 

N280624 

M3 

1-2 

1- 

624 

98 

0.962 

N280625 

M3 

1-2 

1- 

624 

99 

0.986 

N280626 

M3 

1-2 

1- 

624 

98 

0.967 

N286601 

M2 

1-2 

1- 

621 

99 

0.982 

N286602 

M2 

1-3 

1- 

621 

100 

0.994 

N286603 

M2 

1-2 

1- 

621 

99 

0.984 

' Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  as  key)  and  wrong. 
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Table  1-18 

Score  Range,  Percent  Agreement,  and  Cohen’s  Kappa' 
for  the  Dichotomously  Scored  Constructed-Response  Mathematics  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  17 


Item 

Block 

Range  of 
Response 
Codes 

Correct 

Response 

Codes 

Sample 

Size 

Percent 

Agreement 

Cohen’s 

Kanna 

N251101 

Ml 

1-2 

1- 

616 

100 

0.996 

N255801 

M2 

1-2 

1- 

616 

100 

0.986 

N256001 

M3 

1-2 

1- 

564 

99 

0.984 

N256101 

Ml 

1-2 

1- 

616 

100 

1.000 

N259001 

M2 

1-2 

1- 

616 

100 

0.994 

N260601 

Ml 

1-2 

1- 

616 

100 

1.000 

N260801 

M2 

1-2 

1- 

616 

100 

0.997 

N263001 

Ml 

1-2 

1- 

616 

100 

0.994 

N263101 

M2 

1-2 

1- 

616 

100 

0.996 

N264301 

Ml 

1-2 

1- 

616 

100 

0.992 

N264321 

M3 

1-2 

1- 

564 

99 

0.990 

N264521 

M3 

1-2 

1- 

564 

100 

0.996 

N267921 

M3 

1-2 

1- 

564. 

99 

0.985 

N276821 

M3 

1-2 

1- 

564 

100 

0.940 

N276822 

M3 

1-2 

1- 

564 

100 

1.000 

N276823 

M3 

1-2 

1- 

564 

100 

1.000 

N278501 

Ml 

1-2 

1- 

616 

100 

1.000 

N278502 

Ml 

1-2 

1- 

616 

100 

0.991 

N278503 

Ml 

1-2 

1- 

616 

100 

1.000 

N280401 

M2 

1-2 

1- 

616 

100 

0.996 

N280621 

M3 

1-2 

1- 

564 

100 

1.000 

N280622 

M3 

1-2 

1- 

564 

100 

1.000 

N280623 

M3 

1-2 

1- 

564 

100 

1.000 

N280624 

M3 

1-2 

1- 

564 

100 

0.995 

N280625 

M3 

1-2 

1- 

564 

100 

0.996 

N280626 

M3 

1-2 

1- 

564 

100 

0.997 

N285321 

M3 

1-2 

1- 

564 

99 

0.990 

N287301 

Ml 

1-2 

1- 

616 

100 

0.997 

N287302 

Ml 

1-2 

1- 

616 

100 

0.997 

‘ Cohen’s  Kappa  is  a measure  of  reliability  that  is  appropriate  for  items  that  are  dichotomized.  These  items  are  dichotomized  into 
right  (as  indicated  by  key)  and  wrong. 
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Table  1-19 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Const ructed-Response  Writing  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  9 


Item 

Block 

Range  of 
Response 
Codes 

Sample 

Size 

Percent 

A£reement 

Intraclass 

Correlation 

N000602 

BE 

1-3 

459 

97 

0.976 

N000902 

BG 

1-4 

522 

96 

0.967 

N001002 

BG 

1-4 

384 

94 

0.945 

N007602 

BG 

1-4 

263 

93 

0.920 

N007608 

BV 

1-4 

159 

55 

0.819 

NO 14702 

BC 

1-6 

511 

96 

0.962 

NO 14802 

BE 

1-4 

524 

93 

0.957 

NO 14808 

BE 

1-6 

366 

60 

0.863 

Table  1-20 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Writing  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  13 


Item 

Block 

Range  of 
Response 
Codes 

Sample 

Size 

Percent 

Agreement 

Intraclass 

Correlation 

N000302 

BC 

1-4 

591 

92 

0.916 

N000371 

BC 

1-6 

336 

59 

0.846 

N000402 

BD 

1-4 

566 

92 

0.911 

N000471 

BD 

1-6 

329 

53 

0.813 

N000502 

BE 

1-4 

610 

89 

0.893 

N000602 

BE 

1-3 

591 

97 

0.958 

N000902 

BG 

1-4 

616 

93 

0.936 

N001002 

BG 

1-4 

547 

94 

0.930 

890 


Table  1-21 

Score  Range,  Percent  Agreement,  and  Intraclass  Correlation 
for  the  Polytomously  Scored  Constructed-Response  Writing  Items 
Used  in  Long-Term  Trend  Assessment  Scaling,  Age  Class  17 


Item 

Block 

Range  of 
Response 
Codes 

N000302 

BC 

1-4 

N000371 

BC 

1-6 

N000402 

BD 

1-4 

N000471 

BD 

1-4 

N001002 

BG 

1-4 

NO 18002 

BE 

1-4 

NO 19002 

BE 

1-4 

N021002 

BG 

1-4 

Sample 

Percent 

Intraclass 

Size 

Aereement 

Correlation 

501 

92 

0.925 

324 

54 

0.815 

477 

92 

0.915 

307 

50 

0.820 

473 

93 

0.931 

519 

86 

0.904 

489 

93 

0.937 

527 

88 

0.912 

900 
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Appendix  J 


DIFFERENTIAL  ITEM  FUNCTIONING  (DIF)  RESULTS 


Table  J-1 

1996  Mathematics  Items  Identified  as  “C”  or  “CC”  Items  in  At  Least  One  Comparison 


Item 

Block 

Scale 

Category 

Grade 

Comparison 

Group 

Favored 

M077701 

MQ7 

Estimation 

C 

4 

White/Black 

White 

M067501 

MEllA 

Data  Analysis,  Statistics,  and 
Probability 

CC 

8 

White/Black 

White 

M072901 

MN2A 

Data  Analysis,  Statistics,  and 
Probability 

CC 

8 

White/Hispanic 

White 

M075401 

M03A 

Measurement 

CC 

8 

White/Black 

White 

M075601 

M05A 

Measurement 

CC 

8 

White/Black 

White 

M085010 

MVlOA 

Theme:  Measurement 

CC 

8 

White/Hispanic 

White 

M070702 

ME2A 

Algebra  and  Functions 

C 

12 

White/Black 

Black 

M070702 

ME2A 

Algebra  and  Functions 

C 

12 

White/Hispanic 

Hispanic 

M071701 

MG8A 

Algebra  and  Functions 

CC 

12 

White/Black 

Black 

M083701 

MS2A 

Advanced  Data  Analysis, 
Statistics,  and  Probability 

C 

12 

Male/Female 

Male 

‘ For  each  grade  for  which  an  item  was  administered,  three  comparisons  were  performed:  Male/Female,  White/Black,  and 
White/Hispanic. 


Table  J-2 

1996  Science  Items  Identified  as  “C”  or  “CC’  Items  in  At  Least  One  Comparison 


Item 

Block 

Scale 

Category 

Grade 

Comparison 

Group  Favored 

K031202 

SE2A 

Earth  Science 

CC 

4 

Male/Female 

Male 

K031203 

SE3A 

Physical  Science 

CC 

4 

Male/Female 

Female 

K031204 

SE4A 

Physical  Science 

CC 

4 

Male/Female 

Female 

K031304 

SF9A 

Physical  Science 

CC 

4 

White/Black 

White 

K031403 

SG3A 

Earth  Science 

CC 

4 

White/Black 

White 

K032901 

SK3 

Physical  Science 

c 

4 

Male/Female 

Male 

K033502 

SKIOA 

Life  Science 

CC 

4 

White/Black 

Black 

K033503 

SKUA 

Life  Science 

CC 

4 

White/Black 

Black 

K035801 

SN5A 

Physical  Science 

CC 

4 

Male/Female 

Male 

K036301 

SNIOA 

Physical  Science 

CC 

4 

Male/Female 

Male 

K039101 

ST8 

Earth  Science 

c 

4 

Male/Female 

Male 

K039201 

ST9A 

Physical  Science 

CC 

4 

Male/Female 

Male 

K040001 

SU6A 

Physical  Science 

CC 

4 

Male/Female 

Male 

K040101 

SU7 

Physical  Science 

c 

4 

Male/Female 

Male 

K036101 

SN7A 

Earth  Science 

CC 

8 

Male/Female 

Male 

K036401 

SNIOA 

Earth  Science 

CC 

8 

Male/Female 

Male 

K037501 

S06A 

Physical  Science 

CC 

8 

Male/Female 

Male 

* For  each  grade  for  which  an  item  was  administered,  three  comparisons  were  performed:  Male/Female,  White/Black, 
and  White/Hispanic. 

(continued) 
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Table  J-2  (continued) 

1996  Science  hems  Identified  as  “C”  or  “CC  Items  in  At  Least  One  Comparison^ 


Item 

Block 

Scale 

Category 

Grade 

Comparison 

Group  Favored 

K037601 

S07A 

Earth  Science 

CC 

8 

White/Hispanic 

White 

K038201 

S015A 

Earth  Science 

CC 

8 

White/Hispanic 

Hispanic 

K040608 

SC2G 

Physical  Science 

CC 

8 

White/Black 

Black 

K041202 

SG12A 

Earth  Science 

CC 

8 

Male/Female 

Male 

K041601 

SJ2 

Earth  Science 

C 

8 

Male/Female 

Male 

K042602 

SJ15A 

Life  Science 

CC 

8 

Male/Female 

Female 

K042701 

SKI 

Earth  Science 

C 

8 

Male/Female 

Male 

K043101 

SK7A 

Earth  Science 

CC 

8 

Male/Female 

Male 

K043301 

SKll 

Earth  Science 

C 

8 

Male/Female 

Male 

K044101 

ST5A 

Earth  Science 

CC 

8 

Male/Female 

Male 

K044901 

ST13A 

Life  Science 

CC 

8 

White/Black 

Black 

K045001 

ST14A 

Life  Science 

CC 

8 

White/Black 

Black 

K045001 

ST14A 

Life  Science 

CC 

8 

White/Hispanic 

Hispanic 

K046701 

SU16A 

Life  Science 

CC 

8 

White/Hispanic 

White 

K049401 

SM13A 

Earth  Science 

CC 

8 

White/Black 

Black 

K049401 

SM13A 

Earth  Science 

CC 

8 

Male/Female 

Male 

K040805 

SE4A 

Earth  Science 

CC 

12 

White/Hispanic 

Hispanic 

K048901 

SM8A 

Life  Science 

CC 

12 

Male/Female 

Female 

K048901 

SM8A 

Life  Science 

CC 

12 

White/Black 

White 

K048901 

SM8A 

Life  Science 

CC 

12 

White/Hispanic 

White 

K049301 

SM12A 

Life  Science 

CC 

12 

White/Hispanic 

Hispanic 

K049603 

SD3A 

Physical  Science 

CC 

12 

Male/Female 

Female 

K049702 

SF2A 

Physical  Science 

CC 

12 

White/Hispanic 

Hispanic 

K049802 

SG2A 

Earth  Science 

CC 

12 

White/Black 

Black 

K050501 

SJ6A 

SKIO 

Earth  Science 

CC 

12 

White/Black 

White 

K052201 

SSIO 

Life  Science 

c 

12 

Male/Female 

Male 

K052502 

SK15A 

Life  Science 

CC 

12 

White/Hispanic 

Hispanic 

K053601 

SN13A 

SS5A 

Life  Science 

CC 

12 

Male/Female 

Female 

K053701 

SN14A 

Life  Science 

CC 

12 

Male/Female 

Male 

K054005 

S05A 

Life  Science 

CC 

12 

White/Black 

Black 

K058201 

ST13A 

Physical  Science 

CC 

12 

White/Hispanic 

White 

K058301 

ST14A 

Physical  Science 

CC 

12 

Male/Female 

Male 

K058401 

ST15A 

Earth  Science 

CC 

12 

Male/Female 

Male 

K059901 

SU14A 

Earth  Science 

CC 

12 

White/Black 

White 

' For  each  grade  for  which  an  item  was  administered,  three  comparisons  were  performed:  Male/Female,  White/Black,  and 
White/Hispanic. 


Appendix  K 


CORRECTION  OF  THE  NAEP  PROGRAM  DOCUMENTATION  ERROR 
IN  THE  1992  STATE  MATHEMATICS  RESULTS 

Frank  Jenkins  and  Ed  Kulick 
Educational  Testing  Service 


In  April  1995,  results  from  the  1994  Trial  State  Assessment  in  reading  were  released  as  part  of 
the  report  1994  NAEP  Reading:  A First  Look  (Williams,  Reese,  Campbell,  Mazzeo,  & Phillips,  1995). 
Subsequently,  ETS/NAEP  research  scientists  discovered  an  error  in  the  documentation  for  the  ETS 
version  of  the  PARSCALE  program,  which  is  used  to  compute  NAEP  scale  score  results.  The  error 
affected  how  omitted  responses  were  treated  in  the  IRT  scaling  of  the  extended  constructed-response 
items  that  received  partial-credit  scoring  (i.e.,  could  have  several  partially  correct  categories)  in  analyses 
of  1992  and  1994  assessment  data.  The  error  affected  only  polytomous  items;  omitted  multiple-choice 
and  omitted  short  constructed  responses  were  treated  appropriately. 

The  conventional  treatment  in  NAEP  subjects  has  been  to  treat  omitted  responses  (blank 
responses  to  an  item  that  are  followed  by  valid  responses  to  items  that  appear  later  in  the  test)  as  the 
lowest  possible  score  category  in  the  production  of  NAEP  scale  scores.  In  contrast,  not-reached 
responses  (blank  responses  that  are  not  followed  by  any  further  student  responses)  are  treated  as  missing 
data.  As  a result  of  the  documentation  error,  for  a number  of  the  partial  credit  (or  polytomous) 
constructed-response  items  and  across  several  subject  areas,  all  blank  responses  (both  omitted  and  not- 
reached  responses)  to  affected  items  were  treated  as  missing — a reasonable  model  for  treating  omits  but 
one  that  does  not  conform  to  the  conventional  practice  in  NAEP. 

The  error  occurred  because  of  a documentation  error  in  the  description  of  one  of  the  PARSCALE 
control  parameters,  designated  as  POMIT.  The  program  permits  the  analyst  to  choose  two  different  ways 
of  treating  blank  responses  for  partial  credit  items:  (a)  as  missing  data,  and  (b)  as  incorrect,  i.e.  a valid 
response  falling  in  the  lowest  score  category.  The  documentation  indicates  that  by  setting  POMIT=  —1, 
the  treatment  in  (a)  occurs.  By  setting  POMIT=0  or  POMIT=l,  the  treatment  in  (b)  is  supposed  to  occur. 
The  POMIT=l  setting  is  the  program  default.  In  reality,  POMIT=l  and  POMIT=  -1  operate  equivalently, 
treating  blank  responses  as  missing  data. 

The  error  appears  to  have  been  introduced  in  1992  when  the  programs  BILOG  and  PARSCALE 
were  merged  to  form  the  ETS  version  of  PARSCALE.  Verification  of  the  accuracy  of  existing 
documentation,  modifications  to  internal  program  diagnostics,  and  more  systematic  testing  procedures 
for  any  and  all  changes  to  NAEP-related  programs  were  implemented  immediately  to  reduce  the 
likelihood  of  experiencing  this  kind  of  error  in  subsequent  NAEP  cycles. 

The  PARSCALE  documentation  error  affected  a number  of  the  NAEP  scales  constructed  since 
1992.  Specifically,  the  1992  national  and  state  mathematics  results  were  affected  by  the  error.  Results 
from  these  two  assessments  have  been  released  to  the  public  in  a number  of  NAEP  publications.  The  data 
has  also  been  available  to  the  public  through  NCES’s  secondary-use  data  files. 
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NCES  and  ETS  felt  that  the  most  technically  correct  plan  of  action  would  be  to  recalculate  all 
affected  NAEP  scales,  no  matter  how  slight  the  change,  and  to  issue  revised  results.  ETS  was  therefore 
instructed  by  NCES  to  recalculate  all  affected  scales. 

In  recomputing  the  outpoints  for  the  achievement  levels,  an  additional  error  (the  information 
weighting  error)  was  discovered  in  the  procedures  used  by  American  College  Testing  (ACT)  in  1992  to 
“map”  the  achievement-level  outpoints  onto  the  NAEP  scale.  The  procedures  contained  an  incorrectly 
derived  formula.  Details  can  be  found  in  Appendix  I of  the  Technical  Report  of  the  NAEP  1994  Trial 
State  Assessment  in  Reading  (Mazzeo,  Allen,  & Kline,  1995).  ACT  used  revised  procedures  with  the 
correct  formula  to  map  the  achievement- level  outpoints  for  the  1994  U.S.  history  and  geography  scales 
and  the  1996  science  scales.  The  error  in  the  procedures  affected  achievement-level  outpoints  for  the 
1990  mathematics  national  assessment  at  all  grade  levels  (grades  4,  8,  and  12)  and  the  1990  Trial  State 
Assessment  in  mathematics,  which  was  only  at  grade  8.  This  error  also  affected  the  1992  mathematics 
results  for  the  nation  and  the  states.  The  information  weighting  error  added  a source  of  error  to  results  in 
addition  to  the  error  associated  problem  with  defining  omits.  Note  that  the  proficiency  estimates  for  1990 
mathematics  are  correct  and  did  not  have  to  be  recalculated;  only  the  outpoints  for  the  achievement 
levels  were  affected.  For  this  reason,  the  achievement  level  almanac  for  1990  state  mathematics  is  not 
included  in  this  appendix. 

The  information  documenting  the  original  analysis  of  the  1992  data  that  appears  in  the  Technical 
Report  of  the  NAEP  1992  Trial  State  Assessment  Program  in  Mathematics  (Johnson,  Mazzeo,  & Kline, 
1993)  is  substantially  in  agreement  with  the  revised  1994  analysis.  The  transformation  constants  for  the 
revised  analysis  are  provided  in  Table  K-1.  The  information  in  the  other  sections  of  the  technical  report 
for  the  1996  state  mathematics  assessment  refer  only  to  the  revised  analysis  of  the  1994  Trial  State 
Assessment  data. 


Table  K-1 

Transformation  Constants  for  the  1992  Trial  State  Assessment  in  Mathematics 


Scale 

Grade  4 
k,  kz 

Grade  8 
k,  kz 

Numbers  and  Operations 

214.59 

34.16 

268.76 

34.60 

Measurement 

221.40 

33.28 

262.82 

43.95 

Geometry 

220.55 

28.59 

260.44 

33.81 

Data  Analysis,  Statistics,  and  Probability 

217.80 

32.66 

264.58 

39.93 

Algebra  and  Functions 

217.91 

29.00 

264.23 

36.13 

Estimation 

205.41 

35.52 

267.14 

28.14 

As  shown  by  Tables  K-3  through  K-5,  all  jurisdictions  had  average  scores  that  were  adjusted 
upward  slightly  as  a result  of  the  revision  of  results.  For  grade  4,  Tables  K-3  and  K-4  indicate  that 
average  scores  increased  from  .9  to  1.5  points  on  the  proficiency  scale.  Accordingly,  average  scores  in  all 
the  percentiles  go  up  in  a similar  fashion.  Since  all  jurisdictions  were  affected  in  a similar  manner,  there 
is  little  change  in  the  ranking  of  the  jurisdictions.  With  regard  to  achievement  level  results.  Tables  K-7 
and  K-8  indicate  that  although  the  average  scale  scores  moved  up  when  revised,  the  percent  of  students 


at  or  above  the  advanced,  proficient  and  basic  achievement  levels  went  down  slightly  as  a result  of  the 
two  revisions.  For  the  advanced  level,  percentages  for  the  jurisdictions  went  down  0 to  .8  percent,  while 
for  the  below  basic  level,  percentages  went  up  .6  to  1 .7  percent.  Since  the  scale  scores  uniformly  moved 
up,  the  shift  in  achievement  level  percentages  must  be  due  to  the  shift  upward  of  the  achievement  level 
cutpoints  as  result  of  the  information  weighting  error  (see  Table  K-2).  Similar  results  are  evident  for 
grade  8. 


Tables  K-5  and  K-6  indicate  that  average  scores  for  jurisdictions  went  up  from  .6  to  1 point  as  a 
result  of  the  revision  of  scale  scores,  with  similar  slight  upward  adjustments  occurring  at  every  listed 
percentile.  This  change  is  somewhat  less  than  what  was  demonstrated  for  grade  4 and  again  the  ranking 
of  the  jurisdictions  did  not  show  much  change.  Tables  K-9  and  K-10  list  the  original  and  revised  eighth- 
grade  results  for  the  achievement-level  percentages.  These  tables  also  list  values  for  1990,  since  the  Trial 
State  Assessment  was  administered  only  at  grade  8 and  not  grade  4 that  year.  The  effects  of  the  revision 
of  the  1992  results  are  similar  to  those  for  grade  4.  Although  revised  means  were  greater  for  every 
jurisdiction,  the  percentage  above  the  three  achievement  levels  were  slightly  smaller  while  percentages 
below  the  basic  achievement  level  were  slightly  larger.  As  with  grade  4,  this  reflects  the  upward  shift  of 
the  achievement-level  cutpoints  as  a result  of  the  information  weighting  error  (see  Table  K-2). 

For  grade  8,  there  is  the  question  of  the  effect  of  the  revisions  on  the  1990  to  1992  trend.  All  of 
the  17  jurisdictions  that  had  significant  trends  in  average  scale  scores  in  the  original  analysis  also  had 
significant  trends  in  the  revised  analysis.  In  addition,  seven  jurisdictions  that  did  not  have  significant 
trends  in  the  original  analysis  had  significant  trends  in  revision.  A number  of  changes  in  the  percent  of 
students  at  or  above  the  achievement  levels  occurred  due  to  the  combination  of  the  revision  of  scale 
scores  and  achievement  level  cutpoints  for  the  1992  data.  As  a result,  more  differences  across  the  two 
years  were  significant.  For  15  jurisdictions,  there  was  an  increase  in  the  percent  of  students  at  or  above 
an  achievement  level  from  1990  to  1992  that  did  not  change  when  the  analysis  was  redone.  There  were 
nine  jurisdictions  where  there  was  a significant  trend  upward  in  the  percent  of  students  at  or  above  an 
achievement  level  only  after  revision.  For  only  one  jurisdiction  was  a trend  no  longer  significant  after 
revision  (the  percent  at  or  above  advanced  for  Michigan). 


Table  K-2 

Original  and  Revised  ACT  Achievement  Level  Cut  Scores  for 
1992  Mathematics 


Basic 

Proficient 

Advanced 

Grade  4 

Original 

211 

248 

280 

Revised 

214 

249 

282 

Grade  8 

Original 

256 

294 

331 

Revised 

262 

299 

333 

In  summary,  while  changes  in  average  scale  score,  in  percent  of  students  at  or  above  an 
achievement  level,  and  in  1990  to  1992  trends  in  these  statistics  did  occur,  the  meaning  of  the  results 
were  constant  for  most  jurisdictions.  An  exception  was  that  seven  jurisdictions  had  significant 
differences  in  mean  scale  score  from  1990  to  1992  that  did  not  appear  in  the  1992  state  reports.  Also,  10 
jurisdictions  had  changes  in  the  percent  of  students  at  or  above  at  least  one  achievement  level. 
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Table  K-3 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  4 Weighted  Percentages  and  Composite  Scale  Means 
Weighted  Means,  Standard  Deviations,  and  Percentiles 
Original  Results 


Alabama 

Arizona 

Arkansas 

California 

Colorado 

Connecticut 


MEAN 

206. 9(  1.6) 
213. 8(  1.1) 
208. 7(  0.9) 
207. 1(  1.6) 
219. 8(  1.0) 
225. 8(  1.2) 


STD  DEV 

32. 0(  0.6) 
31. 3(  0.5) 
30. 9(  0.6) 
36. 6(  0.8) 
31. 2(  0.4) 
32. 1(  0.7) 


lOTH 

165. 3(  1.7) 
171. 8(  2.1) 
167. 1(  1.3) 
157. 6 ( 2.4) 
178. 8(  1.7) 
183. 6 ( 2.2) 


25TH 

184.2  ( 1.5) 
192. 6(  1.7) 
187. 4(  1.1) 
183. 0(  2.7) 
199. 5(  1.0) 
204. 6(  1.6) 


50TH 

206. 8(  2.4) 
215. 4(  1.3) 
210. 0(  1.3) 
209. 3(  1.3) 
221. 0(  1.4) 
227. 0(  1.7) 


75TH 

230. 1(  1.7) 
235. 6(  1.3) 
230. 4(  1.0) 
232. 7(  1.5) 
241. 0(  1.4) 
248. 2(  1.7) 


90TH 

248. 8(  1.8) 
253. 3(  2.3) 
247. 9(  1.5) 
252. 9(  1.9) 
259. 2(  1.4) 
265. 8(  1.3) 


Delaware 

District  of  Columbia 

Florida 

Georgia 

Hawaii 

Idaho 


216. 6(  0.8) 
191. 2(  0.5) 
212. 4(  1.5) 
214. 3(  1.3) 
212. 8(  1.3) 
220. 3(  1.0) 


32. 4(  0.7) 
32. 8(  0.4) 
32. 6(  0.8) 
32. 8(  0.6) 
34. 0(  0.7) 
28. 1(  0.5) 


174. 6 ( 1.5) 
151. 7(  0.7) 
169. 6(  2.5) 
171. 3(  2.0) 
167. 5 ( 1.7) 
183. 4(  1.4) 


193. 7(  1.0) 
169. 0(  0.7) 
191. 2(  2.0) 
191. 8(  1.2) 
190. 1(  1.6) 
201. 7(  2.5) 


216. 4(  1.0) 
189. 5(  0.7) 
213. 6(  1.8) 
215. 2(  1.3) 
214. 1(  1.9) 
221. 9(  0.9) 


239. 4(  1.5) 
210. 9(  0.9) 
234. 5(  1.4) 
237. 3(  1.7) 
236. 8(  1.2) 
240. 0(  0.7) 


258. 9(  1.4) 
233. 2(  2.2) 
253. 6(  2.6) 
256. 8(  2.0) 
256. 3(  2.1) 
255. 6(  1.2) 


Indiana 

Iowa 

Kentucky 

Louisiema 

Maine 

Maryland 


219.7 ( 

1.1) 

28. 3( 

0.5) 

183. 6 ( 

1.2) 

199. 9( 

229. 0( 

1.1) 

29. 6( 

0.5) 

190. 5( 

2.2) 

210. 0( 

213. 6 ( 

1.0) 

29. 8( 

0.6) 

175. 8( 

1.7) 

193. 3 ( 

202. 8( 

1.4) 

32. 2( 

0.9) 

160.4 ( 

2.3) 

181. 3( 

230.7  ( 

1.0) 

28. 3( 

0.7) 

194.0 ( 

1.9) 

212 .2 ( 

216. 1( 

1.3) 

35. 2( 

0.8) 

169. 5( 

2.0) 

191. 5( 

1.7) 

219.7 ( 

1.2) 

239. 3( 

1.2) 

256.2 ( 

1.0) 

1.1) 

230. 5( 

0.8) 

249. 4( 

0.7) 

265. 8( 

1.1) 

1.1) 

213. 1( 

1.1) 

233. 9( 

0.9) 

252. 6( 

1.7) 

1.6) 

203. 1( 

2.0) 

225. 1( 

3.2) 

244. 0( 

1.5) 

1.2) 

231. 9( 

1.8) 

250. 5( 

1.0) 

265. 1( 

1.3) 

2.6) 

217. 6 ( 

2.2) 

241. 1( 

1.2) 

260. 5( 

1.6) 

Massachusetts 

225. 5( 

1.2) 

31. 2( 

0.7) 

184.4 ( 

Michigan 

218. 6( 

1.8) 

32. 9( 

1.0) 

174. 3 ( 

Minnesota 

227. 5( 

0.9) 

31. 1( 

0.6) 

185. 9 ( 

Mississippi 

200. 1( 

1.1) 

31. 8( 

0.6) 

159. 2( 

Missouri 

221. 0( 

1.2) 

30. 3( 

0.7) 

181. 6( 

Nebraska 

224.2  ( 

1.3) 

30. 8( 

0.6) 

183.3 ( 

1-5) 

205. 3( 

1.5) 

227. 0( 

1.3) 

247. 1( 

1.4) 

264. 0( 

1.1) 

3.3) 

197. 9 ( 

2.3) 

221. 1( 

1.8) 

241. 8( 

1.5) 

258. 9( 

1.6) 

3.8) 

207 .7( 

1.2) 

229. 5( 

1.0) 

249. 4( 

0.9) 

265. 8( 

1.0) 

1.7) 

178. 3( 

1.0) 

200.2 ( 

1.3) 

222. 4( 

1.2) 

240.7 ( 

1.8) 

2.9) 

201. 2( 

1.4) 

221. 8( 

1.5) 

242. 1( 

1.2) 

259. 7( 

1.4) 

1.9) 

203 .8( 

1.7) 

225. 8( 

1.1) 

245. 5( 

1.4) 

262. 4( 

1.6) 

New  Heunpshire 

228. 6( 

1.2) 

28. 7( 

0.5) 

191.6 ( 

New  Jersey 

226. 1( 

1.5) 

31. 4( 

0.9) 

184. 6 ( 

New  Mexico 

211. 8 ( 

1.5) 

30. 2( 

0.7) 

172.5 ( 

New  York 

217 .2 ( 

1.3) 

32. 8( 

0.9) 

173.3  ( 

North  Carolina 

211. 4( 

1.1) 

33. 1( 

0.6) 

167.7 ( 

North  Dakota 

227 .6( 

0.8) 

26. 9( 

0.6) 

192.8 ( 

1.6) 

210. 0( 

1.2) 

229. 2( 

1.4) 

248. 6( 

1.3) 

264. 6< 

2.5) 

2.8) 

205. 6( 

1.9) 

227 .9 ( 

1.6) 

248. 3( 

1.1) 

265. 0< 

2.3) 

2.5) 

191. 1( 

1.8) 

212.3 ( 

1.0) 

232. 2( 

1.7) 

250.7 ( 

2.5) 

3.2) 

195. 9( 

1.4) 

218.5 ( 

1.8) 

240. 1< 

1.7) 

258. 4< 

1.6) 

1.6) 

188. 2( 

1.4) 

212.7  ( 

1.4) 

234. 6< 

1.3) 

253. 0( 

1.2) 

2.8) 

210. 2( 

1.9) 

228.3  ( 

0.8) 

246. 3< 

0.8) 

261. 0< 

1.1) 

Ohio 

217. 5 ( 

1.2) 

31. 4( 

0.8) 

177.2  ( 

Oklahoma 

219. 0( 

1.0) 

27. 4( 

0.6) 

183.7 ( 

Pennsyl  vemi  a 

223.2 ( 

1.4) 

31. 4( 

0.7) 

181.0 ( 

Rhode  Islemd 

214. 0( 

1.6) 

32. 0( 

0.9) 

171.9 ( 

South  Carolina 

211. 1( 

1.1) 

31. 8( 

0.6) 

170.7 ( 

Tennessee 

209.4 ( 

1.4) 

30. 8( 

0.6) 

169.3 ( 

2.7) 

196. 4( 

1.5) 

218. 0( 

1.3) 

239. 1( 

2.3) 

257.7  ( 

1.4) 

1.5) 

200. 9( 

1.2) 

219.2 ( 

1.1) 

237. 2( 

1.4) 

253. 8< 

2.2) 

1.8) 

202. 5( 

1.9) 

224. 8( 

1.8) 

245. 7( 

1.6) 

262. 0( 

2.3) 

3.1) 

193. 2 ( 

2.9) 

215.7 ( 

1.8) 

235. 9( 

2.0) 

253.7 ( 

2.2) 

1.3) 

189. 1( 

1.2) 

210. 3( 

1.2) 

233. 3( 

1.4) 

252. 9( 

2.4) 

2.0) 

188. 3( 

1.9) 

210. 3( 

1.7) 

230. 8( 

1.5) 

248. 5< 

1.9) 

Texas 

216. 6( 

1.3) 

31. 3( 

0.8) 

176.6 ( 

Utah 

222. 8( 

1.0) 

29.3  ( 

0.6) 

184. 8 ( 

Virginia 

219. 6( 

1.3) 

32. 6( 

0.7) 

177 .5  ( 

West  Virginia 

213.9 ( 

1.1) 

30. 1( 

0.6) 

174.9 ( 

Wisconsin 

227  .7  ( 

1.1) 

29. 6( 

0.7) 

188. 5( 

Wyoming 

224.2  ( 

1.0) 

27. 2 ( 

0.5) 

189. 1( 

2.2) 

196. 3( 

1.5) 

217 .0( 

1.6) 

237. 9( 

1.6) 

256. 5( 

2.7) 

1.6) 

203.7 ( 

0.9) 

223. 8( 

1.3) 

242. 8( 

0.8) 

259. 8( 

0.9) 

1.4) 

197. 4( 

1.6) 

219. 8 ( 

1.1) 

241. 9( 

1.8) 

261. 7( 

2.8) 

1.5) 

193. 9( 

1.4) 

214. 1( 

1.2) 

234. 3( 

1.4) 

252. 2( 

1.8) 

2.1) 

208. 7( 

1.0) 

229.4 ( 

1.2) 

248. 3( 

1.3) 

264. 2( 

1.3) 

2.1) 

206. 8( 

1.9) 

225. 1( 

1.2) 

242. 9( 

1.3) 

258. 2( 

1.2) 

Guam 

Virgin  islemds 


191. 1(  0.8) 
178. 0(  1.2) 


34. 3(  0.6) 
28. 3(  0.7) 


147. 1(  1.8) 
140. 4(  1.8) 


167. 5(  0.9) 
159. 2(  2.2) 


191. 4(  1.1) 
178. 7(  1.3) 


213. 8(  1.4) 
197. 2 ( 2.2) 


235. 3(  1.3) 
213. 8(  2.4) 


BEST  COPY  AVAILABLE 


Alabama 

Arizona 

Arkansas 

California 

Colorado 

Connecticut: 

Delaware 

District  of  Columbia 

Florida 

Georgia 

Hawaii 

Idaho 

Indiana 

Iowa 

Kentucky 

Louisiana 

Maine 

Maryland 

Massachusetts 

Michigan 

Minnesota 

Mississippi 

Missouri 

Nebraska 

New  Haiqpshire 
New  Jersey 
New  Mexico 
New  York 
North  Carolina 
North  Dakota 

Ohio 

Oklahoma 
Pennsylvania 
Rhode  Island 
South  Carolina 
Tennessee 

Texas 

Utah 

Virginia 
West  Virginia 
Wisconsin 
Wyoming 

Guam 

Virgin  Islands 


o 


Table  K-4 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  4 Weighted  Percentages  and  Composite  Scale  Means 
Weighted  Means,  Standard  Deviations,  and  Percentiles 
Revised  Results 


MEAN 


208. 3( 

1.6) 

215. 3( 

1.1) 

210. 2( 

0.9) 

208.4 ( 

1.6) 

221. 0( 

1.0) 

226. 8( 

1.1) 

217 .9( 

0.8) 

192. 6( 

0.5) 

213.7 ( 

1.5) 

215. 6( 

1.2) 

214. 1( 

1.3) 

221. 6( 

1.0) 

221. 0( 

1.0) 

229. 9( 

1.0) 

215. 0( 

1.0) 

204. 1( 

1.5) 

231. 6( 

1.0) 

217 .3 ( 

1.3) 

226. 6( 

1.2) 

219. 9( 

1.7) 

228. 5( 

0.9) 

201. 8( 

1.1) 

222.2 ( 

1.2) 

225. 3( 

1.2) 

229.7  ( 

1.2) 

227. 1( 

1.5) 

213. 3( 

1.4) 

218.4  ( 

1.2) 

212. 9 ( 

1.1) 

228.7  ( 

0.8) 

218.7 ( 

1.2) 

220.3  ( 

1.0) 

224.3 ( 

1.3) 

215.4  ( 

1.5) 

212. 5 ( 

1.1) 

210. 9( 

1.4) 

217.9 ( 

1.2) 

224.0 ( 

1.0) 

220.8  ( 

1.3) 

215.3  ( 

1.1) 

228.7  ( 

1.1) 

225.4  ( 

0.9) 

192.8  ( 

0.8) 

178.9  ( 

1.2) 

STD 

DEV 

31. 3( 

0.6) 

30.3  ( 

0.5) 

30. 1( 

0.6) 

36. 0( 

0.8) 

30. 3( 

0.4) 

31. 0( 

0.7) 

31. 4( 

0.7) 

32. 6( 

0.4) 

31. 8( 

0.7) 

31. 9( 

0.6) 

33. 2( 

0.6) 

27.2  ( 

0.5) 

27.4  ( 

0.5) 

28. 6( 

0.5) 

28. 9( 

0.6) 

31. 8( 

1.1) 

27. 4( 

0.7) 

34. 3( 

0.8) 

30. 2( 

0.6) 

31. 9( 

1.0) 

30. 0( 

0.5) 

31. 1( 

0.6) 

29. 3( 

0.7) 

29. 7( 

0.6) 

27. 6 ( 

0.5) 

30. 3( 

0.9) 

29. 3( 

0.6) 

32. 0( 

0.9) 

32. 2( 

0.6) 

25. 9( 

0.5) 

30. 5( 

0.8) 

26. 5( 

0.6) 

30. 4( 

0.7) 

31. 0( 

0.9) 

31. 0( 

0.6) 

29. 9( 

0.6) 

30. 3( 

0.8) 

28. 3( 

0.6) 

31. 7( 

0.7) 

29. 3( 

0.5) 

28. 5( 

0.7) 

26. 2( 

0.5) 

33. 7 ( 

0.6) 

28. 8( 

0.7) 

lOTH 


167  .7( 

1.4) 

174. 4( 

1.8) 

169. 7( 

1.5) 

159. 8( 

1.9) 

181. 2( 

1.8) 

185. 8( 

2.2) 

177 .0( 

1.4) 

153. 2( 

0.8) 

171. 7( 

2.1) 

173. 7( 

1.7) 

170. 0( 

1.5) 

185. 6( 

1.9) 

186. 0( 

1.3) 

192.7 ( 

2.3) 

178. 2( 

1.5) 

162. 9( 

2.5) 

196. 2( 

1.9) 

171. 8( 

1.7) 

186.7 ( 

1.2) 

176. 8( 

3.1) 

188.3 ( 

4.1) 

161.7 ( 

1.5) 

184. 0( 

1.7) 

185.7 ( 

1.8) 

193.8 ( 

1.1) 

186. 9( 

3.1) 

175. 0( 

2.9) 

175.7 ( 

2.2) 

170.4  ( 

1.1) 

195. 0( 

2.4) 

179.5 ( 

1.8) 

186.0 ( 

1.7) 

183.4 ( 

1.9) 

174.4 ( 

2.8) 

173. 0( 

1.3) 

171. 8( 

2.3) 

179.0 ( 

2.4) 

187. 0( 

1.6) 

179.8 ( 

1.3) 

177.3  ( 

1.4) 

190.8 ( 

1.9) 

191.3  ( 

1.6) 

149.6 ( 

1.4) 

140.7  ( 

2.3) 

25TH 


186. 2 ( 

1.8) 

194. 6 ( 

2.3) 

189. 5 ( 

1.1) 

184. 6 ( 

2.3) 

201.4  ( 

1.0) 

206.4  ( 

1.3) 

195.7 ( 

1.3) 

170. 7( 

1.1) 

193. 0( 

1.8) 

193.7 ( 

1.4) 

191. 8( 

2.3) 

203. 5 ( 

1.9) 

201.7 ( 

1.3) 

211. 8( 

1.4) 

195. 2( 

1.0) 

183. 3 ( 

2.0) 

213. 8 ( 

1.3) 

193. 4( 

2.2) 

207. 2 ( 

1.5) 

199. 9( 

2.8) 

209. 4( 

1.1) 

180. 5( 

1.4) 

203. 1( 

1.4) 

205. 7( 

1.8) 

211. 7( 

1.2) 

207. 4 ( 

1.7) 

193. 2 ( 

2.1) 

197. 6 ( 

1.0) 

190. 4( 

1.4) 

212. 0( 

1.1) 

198. 2( 

1.5) 

202. 8( 

0.9) 

204. 3( 

1.5) 

195. 3( 

3.0) 

191. 2( 

1.1) 

190. 5( 

2.1) 

198. 3( 

1.4) 

205. 6( 

1.2) 

199. 3( 

1.5) 

195. 8( 

1.6) 

210. 4( 

0.7) 

208. 6( 

1.4) 

169. 6( 

1.2) 

159. 9( 

2.2) 

50TH 

208. 4(  2.2) 
216. 9(  1.3) 
211. 5(  1.2) 
210. 7(  1.5) 
222. 3(  0.9) 
228. 3(  1.5) 

217. 9(  0.8) 
191. 0(  0.6) 
215. 2(  1.5) 
216. 7(  1.4) 
215. 5(  1.3) 
223. 3(  1.2) 

221. 2(  0.9) 
231. 6(  0.9) 
214. 7(  1.0) 
204. 8(  1.4) 
232. 8(  1.3) 
219. 0(  1.5) 

228. 3(  1.4) 
222. 5(  1.7) 
230. 6(  0.9) 
202. 1(  1.6) 
223. 1(  1.4) 
227. 0(  1.3) 

230. 3(  1.3) 
229. 1(  1.5) 
213. 9(  1.2) 
220. 0(  1.7) 
214. 3(  1.3) 
229. 4(  0.8) 

219.4 ( 0.9) 
220. 6(  1.2) 
226. 0(  1.7) 
217. 3(  1.9) 
211. 9(  1.2) 
212. 0(  1.6) 

218. 5(  1.5) 
225. 1(  1.3) 
221. 2(  1.4) 
215. 6(  1.1) 
230. 5(  1.3) 
226. 4(  1.2) 

193. 2(  1.1) 
179. 8(  1.7) 


75TH 


231. 1( 

1.5) 

236. 5( 

0.9) 

231.4 ( 

1.1) 

233. 5( 

1.3) 

241. 7( 

1.2) 

248. 5( 

0.9) 

240. 1( 

1.4) 

211.8 ( 

1.2) 

235.3 ( 

1.4) 

238. 2( 

1.4) 

237. 5( 

1.1) 

240.7 ( 

0.7) 

240. 1( 

1.2) 

249.7 ( 

0.7) 

234. 9( 

1.0) 

226. 4( 

2.4) 

250. 8( 

1.0) 

242. 0( 

1.7) 

247. 6( 

1.6) 

242. 5( 

1.6) 

249. 6( 

1.1) 

223. 8( 

1.2) 

242.7 ( 

1.1) 

246. 0( 

1.3) 

248. 9( 

1.3) 

248. 7( 

1.0) 

233. 3( 

2.1) 

240. 8( 

1.6) 

235. 5( 

1.6) 

246. 7( 

0.8) 

239. 8( 

1.7) 

238. 0( 

1.4) 

246. 1( 

1.8) 

236. 8( 

1.7) 

234. 2( 

1.4) 

231. 9( 

1.4) 

238.7 ( 

1.8) 

243. 3( 

1.0) 

242. 5( 

1.9) 

235. 2( 

1.4) 

248. 6( 

1.2) 

243. 5( 

0.9) 

215. 4( 

1.3) 

198. 3( 

1.2) 

90TH 


249. 3( 

2.4) 

253. 4( 

1.4) 

248. 5( 

1.5) 

253. 1( 

2.6) 

259. 1( 

0.8) 

265. 3( 

1.6) 

258. 8( 

1.4) 

234 .4( 

1.4) 

253. 8( 

3.1) 

256. 8( 

1.2) 

256. 5( 

1.9) 

255. 5( 

1.1) 

256. 3( 

1.0) 

265. 4( 

1.5) 

252. 8( 

1.3) 

244. 5( 

1.4) 

264. 6( 

1.2) 

260. 5( 

1.2) 

263 .6( 

1.7) 

258.7  < 

1.6) 

265. 3( 

1.4) 

241. 6( 

1.2) 

259. 5( 

1.6) 

262. 1( 

1.8) 

264. 1( 

2.3) 

264.6 ( 

1.9) 

250.9  ( 

1.8) 

258.3 ( 

1.0) 

253. 0( 

1.2) 

260. 8 ( 

1.4) 

257.4 ( 

1.1) 

253. 9 ( 

1.9) 

261.7 ( 

1.8) 

253.8  ( 

3.0) 

253. 1( 

1.8) 

248. 8( 

1.4) 

256. 5 ( 

2.3) 

259.8 ( 

1.0) 

261.3 ( 

2.2) 

252.3 ( 

1.6) 

263.6 ( 

1.8) 

258.2 ( 

0.9) 

236.2  ( 

1.7) 

215.0 ( 

2.5) 

best  copy  available 
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Ale^sama 

Arizona 

Arkansas 

California 

Colorado 

Connecticut 

Delaware 

District  of  Columbia 

Florida 

Georgia 

Hawaii 

Idaho 

Indiema 

Iowa 

Kentucky 

Louisiema 

Maine 

Marylemd 

Massachusetts 

Michigem 

Minnesota 

Mississippi 

Missouri 

Nebraska 

New  Hazigpshire 
New  Jersey 
New  Mexico 
New  York 
North  Carolina 
North  Dakota 

Ohio 

Oklahoma 
Pennsy  1 vani  a 
Rhode  Islemd 
South  Carolina 
Tennessee 

Texas 

Utah 

Virginia 

West  Virginia 

Wisconsin 

Wyoming 

Guam 

Virgin  Islands 


Table  K-5 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  8 Weighted  Percentages  and  Composite  Scale  Means 
Weighted  Means,  Standard  Deviations,  and  Percentiles 
Original  Results 


MEAN 


251.3 ( 

1.7) 

264. 6( 

1.3) 

255.4 ( 

1.2) 

260. 1( 

1.7) 

271.7 ( 

1.1) 

273. 1( 

1.1) 

262. 1( 

1.0) 

233. 9( 

0.9) 

259. 1( 

1.5) 

258. 5( 

1.2) 

256. 6( 

0.9) 

274.4  ( 

0.8) 

269.4 ( 

1.2) 

282.8 ( 

1.0) 

261.4 ( 

1.1) 

249. 1( 

1.7) 

278. 0( 

1.0) 

264.2 ( 

1.3) 

272. 1( 

1.1) 

266.6 ( 

1.4) 

281. 8( 

1.0) 

245.5  ( 

1.2) 

270.4 ( 

1.2) 

277. 0( 

1.1) 

277. 6 ( 

1.0) 

271. 2 ( 

1.6) 

258. 8( 

0.9) 

265.7 ( 

2.1) 

257.6  ( 

1.2) 

282. 6( 

1.2) 

267 .4  ( 

1.5) 

267.4  ( 

1.2) 

270.7  ( 

1.5) 

265. 1( 

0.7) 

260. 0( 

1.0) 

258. 0( 

1.4) 

263. 8 ( 

1.3) 

273.6  ( 

0.7) 

267 .!( 

1.2) 

258. 2( 

1.0) 

277. 3 ( 

1.5) 

274 .4  ( 

0.9) 

234.3  ( 

1.0) 

221. 8( 

1.1) 

STD  DEV 

35. 8(  1.2) 
32. 8(  0.7) 
34. 3(  0.6) 
38. 7(  1.1) 
33. 1(  0.6) 
36. 0(  0.9) 

35. 7(  0.7) 
36. 5(  1.0) 
36. 8(  0.8) 
34. 6(  0.6) 
37. 8(  0.7) 
30. 5(  0.5) 

33. 9(  0.6) 
30. 0(  0.6) 
34. 3(  0.6) 
34. 0(  0.9) 
30. 7(  0.8) 
39. 2(  0.9) 

34. 2(  0.7) 
35. 4(  0.6) 
31. 9(  0.5) 
34. 8(  0.6) 
32. 7(  0.7) 
32. 4(  0.6) 

30. 4(  0.7) 
36. 0(  0.9) 
32. 3(  0.7) 
39. 4(  1.3) 
35. 4(  0.8) 
28. 3(  0.6) 

34. 4(  0.9) 
32. 3(  0.6) 
34. 5(  0.9) 
33. 4(  0.5) 
35. 3(  0.7) 
33. 9(  0.7) 

37. 2(  0.7) 
32. 0(  0.8) 
35. 4(  0.7) 
31. 1(  0.6) 
33. 2(  0.7) 
29. 6(  0.5) 

39. 2(  0.8) 
30. 1(  0.6) 


lOTH 


205. 5( 

1.9) 

222. 1( 

1.6) 

210. 9 ( 

1.6) 

208. 8( 

2.7) 

227. 6 ( 

1.6) 

224. 3( 

2.6) 

216.4 ( 

1.8) 

188. 6( 

1.0) 

210. 3( 

3.0) 

213.8 ( 

1.5) 

208. 2( 

1.5) 

234. 9( 

1.1) 

225. 3( 

1.5) 

243. 9 ( 

2.1) 

216.4 ( 

1.7) 

204. 6 ( 

2.6) 

239. 1( 

2.3) 

212.9 ( 

1.8) 

228. 6( 

1.4) 

220. 0( 

1.4) 

240. 0( 

1.4) 

200. 6( 

1.2) 

227. 9 ( 

2.9) 

234. 0( 

1.7) 

237. 9 ( 

1.1) 

222. 2( 

1.9) 

217. 1( 

2.0) 

213.2 ( 

3.1) 

212. 3( 

2.6) 

244.9 ( 

1.2) 

222. 0( 

1.9) 

225.9 ( 

1.3) 

225. 1( 

2.3) 

220. 9( 

1.2) 

215. 2( 

1.3) 

214.0 ( 

2.1) 

215.8 ( 

2.7) 

231. 6( 

1.2) 

221.4 ( 

1.5) 

217. 8 ( 

1.5) 

233. 1( 

2.6) 

236. 5( 

1.0) 

183. 8 ( 

2.1) 

183. 1( 

1.2) 

25TH 


226. 8( 

1.8) 

242. 8( 

1.3) 

232. 9( 

1.2) 

233.7 ( 

2.6) 

250. 1( 

1.2) 

248. 8( 

1.7) 

238.7 ( 

0.9) 

208.7 ( 

1.2) 

234. 0( 

1.6) 

234.7 ( 

1.5) 

230. 9( 

1.0) 

254. 5( 

0.9) 

246. 8( 

1.2) 

262. 5( 

1.4) 

238. 5( 

1.6) 

226. 4( 

2.2) 

258. 2( 

1.2) 

236. 9( 

2.3) 

248. 6( 

2.2) 

243. 1( 

2.3) 

260. 2( 

1.4) 

221. 3( 

1.3) 

248. 5( 

1.8) 

256. 4( 

1.2) 

257.7 ( 

0.8) 

247 .0( 

2.0) 

237  .4 ( 

0.9) 

240. 9( 

2.7) 

233. 6( 

1.3) 

263.7 ( 

1.4) 

244. 4( 

2.0) 

247. 1( 

1.4) 

247. 9 ( 

1.5) 

242. 8( 

1.1) 

235. 2( 

1.1) 

234. 9( 

1.5) 

237. 5( 

1.2) 

252. 8( 

1.7) 

243. 0( 

1.7) 

237. 0( 

1.0) 

256. 7( 

2.1) 

254. 1( 

1.2) 

207. 1( 

1.8) 

201. 2( 

1.6) 

50TH 


250. 6( 

2.0) 

265. 0( 

1.9) 

256. 2( 

1.2) 

261. 4( 

1.8) 

273. 3( 

1.1) 

275. 4( 

0.8) 

262.4 ( 

1.3) 

233. 3( 

1.8) 

260. 1( 

2.0) 

259. 4( 

1.3) 

257. 5 ( 

1.6) 

275. 2( 

1.2) 

269. 8( 

1.3) 

283.7 ( 

1.1) 

262. 0( 

1.0) 

249.7 ( 

1.6) 

278. 6( 

1.1) 

265. 0( 

1.3) 

272. 9( 

2.0) 

268. 3( 

1.6) 

282. 9( 

1.3) 

245.4 ( 

1.2) 

271. 8( 

1.4) 

279. 1( 

1.4) 

278. 0( 

0.9) 

272. 9( 

1.9) 

259. 3( 

1.0) 

268. 3( 

1.8) 

258. 5( 

1.2) 

284.3  ( 

1.0) 

269. 1( 

1.6) 

268.3  ( 

1.1) 

272. 2( 

1.4) 

266.7  ( 

1.2) 

259. 2( 

1.2) 

258.2  ( 

1.6) 

263.9 ( 

1.9) 

274. 9 ( 

0.8) 

267.4 ( 

1.7) 

258.4 ( 

1.7) 

279. 1( 

1.5) 

275. 2 ( 

1.2) 

233.4 ( 

1.6) 

221.4 ( 

1.2) 

75TH 


275.7 ( 

1.7) 

286. 9( 

1.2) 

278. 8( 

1.6) 

287 .9( 

1.7) 

295. 0( 

1.2) 

299 .2( 

1.0) 

286. 6( 

1.5) 

257. 2( 

2.8) 

284. 8( 

1.7) 

282. 9( 

2.1) 

282. 8( 

1.0) 

295.7 ( 

0.8) 

292.7 ( 

1.9) 

303. 9( 

1.5) 

284. 6( 

1.3) 

272. 1( 

2.0) 

299. 0( 

1.7) 

292. 4( 

1.6) 

296. 8( 

1.6) 

291. 7( 

2.9) 

304. 3( 

1.4) 

270. 0( 

1.6) 

292. 8( 

1.6) 

299.7 ( 

1.0) 

298. 5( 

1.1) 

296. 8( 

2.2) 

280. 9( 

1.0) 

292. 7( 

1.4) 

282. 0( 

1.4) 

302. 0( 

1.4) 

291. 6( 

1.4) 

290. 1( 

1.4) 

295. 0( 

1.1) 

288. 5( 

1.7) 

284. 6( 

1.7) 

282. 1( 

1.4) 

289. 4( 

2.3) 

296. 1( 

1.2) 

291. 4( 

1.6) 

280. 5( 

1.1) 

300. 6( 

1.5) 

294. 9( 

1.2) 

261. 4( 

2.5) 

242. 1( 

1.5) 

90TH 


298. 8( 

2.0) 

306. 5( 

1.3) 

298. 6( 

1.6) 

308. 8( 

2.5) 

313. 3( 

1.2) 

318. 4( 

1.4) 

307. 0( 

1.4) 

280. 2( 

1.7) 

306. 5( 

2.0) 

303. 1( 

1.5) 

304.7 ( 

1.3) 

313. 0( 

1.1) 

313. 1( 

2.9) 

320. 6( 

1.6) 

305.4 ( 

2.8) 

293. 0( 

1.8) 

316.4 ( 

1.3) 

313.5 ( 

1.6) 

315.7 ( 

1.7) 

311. 2( 

2.3) 

322. 3( 

1.4) 

290. 9( 

2.0) 

311.5 ( 

1.3) 

316.9 ( 

1.5) 

315. 9( 

2.0) 

317. 1( 

1.6) 

300. 2( 

1.3) 

314. 5( 

2.4) 

302.7 ( 

1.5) 

318. 0( 

1.7) 

310. 4( 

1.5) 

307.7 ( 

1.5) 

314. 5 ( 

1.8) 

306. 9( 

1.1) 

306.7 ( 

1.5) 

301. 6( 

1.5) 

312.0 ( 

1.5) 

313.7 ( 

1.2) 

313. 4( 

1.5) 

298. 2( 

1.8) 

317. 8 ( 

1.4) 

312. 1( 

1.1) 

286. 2( 

3.7) 

260. 2( 

1.6) 

BEST  COPY  AVAILABLE 
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Alabama 

Arizona 

Arkansas 

California 

Colorado 

Cozmecticut 

Delaware 

District  o£  Columbia 

Florida 

Georgia 

Hawaii 

Idaho 

Indiana 

Iowa 

Kentucky 

Louisiana 

Maine 

Maryland 

Massachusetts 

Michigan 

Minnesota 

Mississippi 

Missouri 

Nebraska 

New  Hampshire 
New  Jersey 
New  Mexico 
New  York 
North  Carolina 
North  Dakota 

Ohio 

Oklahoma 
Pennsylvania 
Rhode  Island 
South  Carolina 
Tennessee 

Texas 

Utah 

Virginia 
West  Virginia 
Wisconsin 
Wyoming 

Guam 

Virgin  Islands 


Table  K-6 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  8 Weighted  Percentages  and  Composite  Scale  Means 
Weighted  Means,  Standard  Deviations,  and  Percentiles 
Revised  Results 


MSAN 


252.2 ( 

1.7) 

265. 4( 

1.3) 

256. 3( 

1.2) 

260. 9( 

1.7) 

272.4 ( 

1.0) 

273. 7( 

1.1) 

262. 9( 

1.0) 

234. 9( 

0.9) 

259. 9( 

1.5) 

259. 4( 

1.2) 

257. 4( 

0.9) 

275.1 ( 

0.7) 

270. 1( 

1.1) 

283. 4( 

1.0) 

262. 2( 

1.1) 

250. 0( 

1.7) 

278. 6( 

1.0) 

264. 8( 

1.3) 

272. 8( 

1.0) 

267. 4( 

1.4) 

282. 4( 

1.0) 

246. 5( 

1.2) 

271. 1( 

1.2) 

277. 7( 

1.1) 

278. 2( 

1.0) 

271. 9( 

1.6) 

259. 6( 

0.9) 

266. 4( 

2.1) 

258. 4( 

1.2) 

283. 2( 

1.1) 

268. 1( 

1.5) 

268. 1( 

1.1) 

271. 4( 

1.5) 

265. 9( 

0.7) 

260. 8( 

1.0) 

258. 8( 

1.4) 

264. 6( 

1.3) 

274. 3( 

0.7) 

267. 9( 

1.2) 

259. 1( 

1.0) 

277. 9 ( 

1.5) 

275. 1( 

0.9) 

235. 1( 

1.0) 

222. 8( 

1.1) 

STD 

DEV 

35. 5( 

1.2) 

32. 4( 

0.6) 

33. 9( 

0.6) 

38. 3( 

1.1) 

32. 7( 

0.6) 

35. 6( 

0.9) 

35. 3( 

0.7) 

36. 2( 

0.9) 

36. 4( 

0.8) 

34.3  ( 

0.6) 

37.3  ( 

0.6) 

30. 1( 

0.5) 

33. 5( 

0.6) 

29. 5( 

0.6) 

33. 9( 

0.6) 

33. 7( 

0.8) 

30. 3( 

0.8) 

38. 9( 

0.9) 

33. 8( 

0.7) 

35. 0( 

0.6) 

31. 5( 

0.5) 

34. 5( 

0.6) 

32. 3( 

0.7) 

32. 0( 

0.5) 

30. 0( 

0.7) 

35. 6( 

0.9) 

31. 9( 

0.7) 

39. 0( 

1.3) 

35. 1( 

0.8) 

27. 9 ( 

0.6) 

34. 2( 

0.9) 

31. 9( 

0.6) 

34. 1( 

0.9) 

33. 1( 

0.5) 

34. 9( 

0.7) 

33. 5( 

0.7) 

36. 8( 

0.7) 

31. 6( 

0.8) 

35. 0( 

0.6) 

30. 8( 

0.6) 

32. 7( 

0.7) 

29. 2( 

0.5) 

39. 2( 

0.9) 

30. 0( 

0.6) 

lOTH 


206. 8( 

1.8) 

223. 3( 

2.3) 

212. 1( 

1.7) 

210. 1( 

1.8) 

228. 8( 

1.8) 

225. 4( 

2.6) 

217. 7( 

2.4) 

189. 9 ( 

1.1) 

211. 5( 

2.9) 

215. 0( 

1.4) 

209. 4( 

2.5) 

236. 1( 

1.1) 

226. 4( 

1.4) 

245. 0( 

2.6) 

217. 6 ( 

1.8) 

205. 8( 

2.7) 

240. 3( 

2.0) 

214. 1( 

2.6) 

229. 7( 

1.4) 

221. 2( 

1.1) 

241. 1( 

1.7) 

201. 8( 

1.1) 

229. 0( 

2.3) 

235. 2( 

1.8) 

239. 0( 

1.3) 

223. 4( 

2.0) 

218. 4( 

2.7) 

214. 6 ( 

3.3) 

213. 4 ( 

2.1) 

246. 0( 

1.3) 

223. 1( 

2.3) 

227. 0( 

1.1) 

226. 2( 

2.5) 

222.0 ( 

0.9) 

216. 4( 

1.6) 

215. 3( 

2.2) 

217. 0( 

2.0) 

232. 7( 

1.3) 

222. 5 ( 

1.7) 

219. 0( 

1.8) 

234. 3 ( 

2.3) 

237. 6 ( 

1.1) 

184. 5 ( 

1.9) 

184. 2 ( 

1.7) 

25TH 


227.9 ( 

1.9) 

243. 9 ( 

1.4) 

234. 1( 

1.4) 

234. 8( 

2.8) 

251. 1( 

1.1) 

249. 8( 

1.4) 

239. 7( 

1.2) 

210. 0( 

1.2) 

235. 1( 

1.7) 

235. 8( 

1.2) 

232. 0( 

0.9) 

255. 4( 

1.0) 

247. 8 ( 

1.5) 

263. 4( 

1.2) 

239. 5 ( 

1.6) 

227. 4 ( 

1.9) 

259. 1( 

1.1) 

237. 9 ( 

2.1) 

249. 6( 

2.2) 

244. 1( 

2.1) 

261. 1( 

1.2) 

222. 5 ( 

1.1) 

249. 6 ( 

1.9) 

257.3 ( 

1.2) 

258. 6( 

0.7) 

248.0 ( 

2.3) 

238. 5( 

1.1) 

241.9 ( 

2.5) 

234.6 ( 

1.8) 

264. 6( 

1.3) 

245. 4( 

1.8) 

248. 1( 

1.3) 

248. 9( 

1.3) 

243. 8 ( 

1.2) 

236. 3( 

1.2) 

236. 0( 

1.5) 

238. 6( 

1.3) 

253. 8( 

1.3) 

244.0 ( 

1.7) 

238. 1( 

1.1) 

257. 6 ( 

2.1) 

255. 1( 

1.1) 

208. 1( 

3.3) 

202. 2( 

1.8) 

50TH 


251. 6( 

1.7) 

265. 9( 

1.8) 

257. 2( 

1.2) 

262. 3( 

1.4) 

274. 1( 

1.2) 

276. 1( 

0.9) 

263. 3( 

1.2) 

234. 3( 

1.9) 

260. 9( 

1.9) 

260. 3( 

1.3) 

258. 3( 

1.4) 

276. 0( 

1.0) 

270. 6( 

1.5) 

284. 3( 

1.3) 

262. 9( 

1.1) 

250. 6( 

1.8) 

279. 4( 

1.3) 

265. 9( 

0.9) 

273. 7( 

1.3) 

269. 1( 

1.4) 

283. 6( 

1.4) 

246. 4( 

1.3) 

272. 6 ( 

1.3) 

279. 8( 

1.5) 

278. 7( 

0.8) 

273. 6 ( 

1.8) 

260. 1( 

0.9) 

269. 2( 

1.9) 

259. 3( 

0.9) 

284. 9( 

1.0) 

269. 9( 

1.5) 

269. 1( 

1.1) 

273. 0( 

1.3) 

267. 6( 

1.1) 

260. 0( 

1.3) 

259. 1( 

1.6) 

264. 8 ( 

1.6) 

275. 7( 

0.9) 

268. 2( 

1.7) 

259. 4( 

1.4) 

279. 8( 

1.3) 

276.0 ( 

1.2) 

234. 3( 

1.9) 

222. 4( 

1.5) 

75TH 


276. 5( 

1.7) 

287. 6 ( 

1.1) 

279. 5( 

1.4) 

288. 5( 

2.2) 

295. 4( 

1.1) 

299. 6( 

0.8) 

287. 2( 

1.7) 

258. 1( 

2.1) 

285. 5( 

1.5) 

283. 6 ( 

2.1) 

283. 4( 

1.0) 

296. 1( 

0.9) 

293. 2( 

1.5) 

304. 2( 

1.8) 

285. 2( 

1.4) 

272.9 ( 

1.8) 

299. 4( 

1.5) 

292.9 ( 

1.2) 

297. 2( 

1.6) 

292.3 ( 

2.3) 

304. 5( 

1.3) 

270. 8( 

1.5) 

293. 3( 

1.7) 

300. 1( 

1.3) 

298.9 ( 

1.3) 

297.2 ( 

2.0) 

281. 6( 

1.1) 

293.3  ( 

1.5) 

282. 6( 

1.2) 

302.3 ( 

1.5) 

292. 2( 

1.0) 

290. 7( 

1.5) 

295. 4( 

1.2) 

289. 1( 

1.5) 

285. 2( 

1.7) 

282.8 ( 

1.8) 

290. 0( 

1.9) 

296. 6( 

1.2) 

291. 9( 

1.6) 

281. 1( 

0.9) 

300. 9( 

1.4) 

295. 4( 

1.0) 

262.4  ( 

1.7) 

243.0 ( 

1.6) 

90TH 


299. 2( 

2.3) 

306. 8( 

1.2) 

299. 0( 

1.2) 

309. 0( 

2.2) 

313. 5 ( 

1.1) 

318. 4( 

1.3) 

307. 3( 

1.5) 

280. 8( 

1.6) 

306. 7( 

1.8) 

303. 3( 

1.3) 

305. 0( 

1.6) 

313. 1( 

1.1) 

313.4 ( 

3.2) 

320. 6( 

1.3) 

305. 6( 

2.3) 

293. 5( 

1.5) 

316. 5( 

1.7) 

313. 6( 

1.9) 

315. 7( 

1.5) 

311. 4( 

2.2) 

322. 2( 

1.6) 

291. 4( 

1.3) 

311. 6( 

1.4) 

317.0 ( 

1.2) 

316. 0( 

1.7) 

317. 0( 

2.1) 

300. 6( 

1.0) 

314. 6( 

2.0) 

302. 9( 

2.0) 

317. 9( 

1.6) 

310. 6( 

1.6) 

308. 1( 

1.3) 

314. 6( 

1.9) 

307. 1( 

1.1) 

307. 0( 

1.7) 

301. 9( 

1.7) 

312. 0( 

2.1) 

313. 9( 

1.2) 

313. 5( 

1.8) 

298. 6( 

1.5) 

317. 9( 

1.2) 

312. 2( 

1.3) 

286. 7( 

1.8) 

260. 9( 

2.0) 
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best  copy  available 


Table  K-7 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  4 Weighted  Percentages  and  Composite  Scale,  Means 
Percent  of  Students  At  or  Above  the  Achievement  Levels 
Original  Results 


Alabama 

Arizona 

Arkansas 

California 

Colorado 

Connecticut 


N WEIGHTED  PCT  [CV] 


MEAN 


ADVANCED 


PR7CIENT 


BASIC 


< BASIC 


2605 

1.9( 

0.1) 

[ 

3\] 

206. 9( 

1.6) 

0.6( 

2741 

1.8( 

0.0) 

t 

2\] 

213. 8 ( 

1.1) 

1.2( 

2621 

1.2( 

0.0) 

t 

4\] 

208. 7( 

0.9) 

0.6( 

2412 

12.3  ( 

0.3) 

t 

3\] 

207. 1( 

1.6) 

1.6( 

2906 

1.7( 

0.1) 

t 

3\] 

219. 8( 

1.0) 

2.K 

2600 

1.2( 

0.0) 

t 

3\] 

225. 8( 

1.2) 

3.7( 

0.2) 

10. 5( 

1.3) 

44. 7 ( 

2.2) 

55. 3( 

2.2) 

0.3) 

13. 5( 

0.9) 

55. 2( 

1.7) 

44. 8( 

1.7) 

0.2) 

10.0  ( 

0.8) 

48. 8( 

1.3) 

51. 2( 

1.3) 

0.5) 

12. 7 ( 

1.2) 

48. 0( 

2.0) 

52. 0( 

2.0) 

0.4) 

18. 2( 

1.1) 

62. 5 ( 

1.4) 

37. 5( 

1.4) 

0.6) 

25.2  ( 

1.4) 

68. 8( 

1.5) 

31. 2( 

1.5) 

Delaware 

2040 

0.3( 

District  of  Columbia 

2399 

0.2( 

Florida 

2828 

5.4( 

Georgia 

2766 

3.4( 

Hawaii 

2625 

0.5( 

Idaho 

2784 

0.6( 

0.0) 

t 

0\] 

216. 6( 

0.8) 

2.3( 

0.4) 

0.0) 

t 

0\] 

191. 2( 

0.5) 

1.0( 

0.2) 

0.2) 

t 

3\] 

212. 4( 

1.5) 

1.5( 

0.4) 

0.1) 

t 

4\] 

214.3 ( 

1.3) 

1.5( 

0.4) 

0.0) 

t 

2\] 

212. 8 ( 

1.3) 

1.5( 

0.4) 

0.0) 

t 

3\] 

220. 3( 

1.0) 

l.K 

0.3) 

17. 0( 

0.8) 

56. 2( 

1.0) 

43. 8( 

1.0) 

5.5( 

0.3) 

24. 9( 

1.0) 

75. 1( 

1.0) 

13. 5 ( 

1.4) 

53. 4( 

2.0) 

46. 6( 

2.0) 

15. 7( 

1.2) 

54. 9( 

1.7) 

45. 1( 

1.7) 

15.2  ( 

1.0) 

53. 7( 

1.8) 

46. 3( 

1.8) 

16. 1( 

1.1) 

64. 2( 

1.7) 

35. 8( 

1.7) 

Indiana 

2593 

2.7( 

0.1) 

t 

3\] 

219. 7( 

Iowa 

2770 

1.3( 

0.0) 

t 

4\] 

229. 0( 

Kentucky 

2703 

1.8( 

0.1) 

t 

4\] 

213. 6( 

Louisiana 

2792 

2.2  ( 

0.1) 

t 

3\] 

202. 8( 

Maine 

1898 

0.6( 

0.0) 

t 

3\] 

230. 7( 

Maryland 

2844 

2.0( 

0.1) 

t 

3\] 

216. 1( 

Massachusetts 

2549 

2.K 

0.1) 

t 

4\] 

225. 5( 

Michigan 

2412 

4.K 

0.1) 

t 

4\] 

218. 6( 

Minnesota 

2640 

2.K 

0.1) 

t 

4\] 

227. 5( 

Mississippi 

2712 

1.4( 

0.0) 

t 

3\] 

200. 1( 

Missouri 

2509 

2.K 

0.1) 

t 

5\] 

221. 0( 

Nebraska 

2327 

0.7( 

0.0) 

t 

3\] 

224.2 ( 

New  Hampshire 

2265 

0.5( 

0.0) 

t 

3\] 

228. 6( 

New  Jersey 

2231 

2.8( 

0.1) 

t 

4\] 

226. 1( 

New  Mexico 

2342 

0.8( 

0.1) 

t 

6\] 

211. 8( 

New  York 

2284 

6.7( 

0.2) 

t 

3\] 

217.2 ( 

North  Carolina 

2884 

3.0( 

0.1) 

t 

3\] 

211.4 ( 

North  Dakota 

2193 

0.3( 

0.0) 

t 

4\] 

227.6 ( 

Ohio 

2637 

4.9( 

0.1) 

t 

3\] 

217. 5( 

Oklahoma 

2254 

1.6( 

0.0) 

t 

3\] 

219. 0( 

Pennsylvania 

2740 

4.6( 

0.2) 

t 

4\] 

223. 2( 

Rhode  Island 

2390 

0.4( 

0.0) 

t 

5\] 

214. 0( 

South  Carolina 

2771 

1.8( 

0.1) 

t 

3\] 

211. 1( 

Tennessee 

2708 

2.4( 

0.1) 

t 

3\] 

209. 4( 

Texas 

2623 

9.0( 

0.3) 

t 

4\] 

216. 6( 

Utah 

2799 

1.3( 

0.0) 

t 

2\] 

222. 8( 

Virginia 

2786 

2.8( 

0.1) 

t 

3\] 

219. 6( 

West  Virginia 

2786 

0.9( 

0.0) 

t 

4\] 

213. 9( 

Wisconsin 

2780 

2.K 

0.1) 

t 

A\] 

227. 7( 

Wyoming 

2605 

0.3( 

0.0) 

t 

3\] 

224.2 ( 

1.1) 

1.5( 

0.3) 

16.2  ( 

1.1) 

61. 6( 

1.6) 

38. 4( 

1.6) 

1.1) 

3.K 

0.5) 

27. 0( 

1.3) 

74. 1( 

1.4) 

25. 9( 

1.4) 

1.0) 

1.4( 

0.5) 

12. 9 ( 

1.1) 

52. 7( 

1.5) 

47.3  ( 

1.5) 

1.4) 

0.6( 

0.2) 

7.9( 

0.8) 

40. 5( 

2.0) 

59. 5( 

2.0) 

1.0) 

3.0( 

0.6) 

28.3  ( 

1.5) 

76. 3( 

1.3) 

23. 7( 

1.3) 

1.3) 

2.8( 

0.4) 

18. 9 ( 

1.2) 

56. 8( 

1.6) 

43. 2( 

1.6) 

1.2) 

3.K 

0.5) 

24. 0( 

1.5) 

69. 7( 

1.6) 

30. 3( 

1.6) 

1.8) 

1.7( 

0.5) 

19. 1( 

1.7) 

62. 1( 

2.2) 

37.9  { 

2.2) 

0.9) 

3.4( 

0.5) 

26. 8( 

1.2) 

71. 8( 

1.4) 

28. 2( 

1.4) 

1.1) 

0.4( 

0.1) 

6.5( 

0.7) 

37. 3( 

1.3) 

62.7  ( 

1.3) 

1.2) 

1.9( 

0.3) 

19. 3( 

1.3) 

63. 6 ( 

1.6) 

36. 4( 

1.6) 

1.3) 

2.6( 

0.5) 

22. 5( 

1.7) 

68. 1( 

1.8) 

31. 9( 

1.8) 

1.2) 

3.0( 

0.6) 

25. 7( 

1.7) 

73. 8( 

1.6) 

26. 2( 

1.6) 

1.5) 

3.2( 

0.7) 

25.3  ( 

1.6) 

69. 9( 

2.1) 

30. 1( 

2.1) 

1.5) 

1.0( 

0.4) 

11.4  ( 

1.3) 

51. 6( 

1.9) 

48. 4( 

1.9) 

1.3) 

2.0( 

0.3) 

17. 4 ( 

1.3) 

58. 8( 

1.9) 

41. 2( 

1.9) 

1.1) 

1.6( 

0.4) 

13. 2( 

0.9) 

52. 1( 

1.6) 

47.9  ( 

1.6) 

0.8) 

1.8( 

0.3) 

22. 9( 

1.1) 

74. 2( 

1.2) 

25. 8( 

1.2) 

1.2) 

1.9( 

0.3) 

16. 8( 

1.1) 

58. 8( 

1.7) 

41. 2( 

1.7) 

1.0) 

1.2( 

0.4)  . 

14.4  ( 

1.1) 

61. S( 

1.6) 

38. 5( 

1.6) 

1.4) 

2.6( 

0.5) 

22. 5( 

1.5) 

66. 4( 

1.9) 

33.6  ( 

1.9) 

1.6) 

1.6( 

0.4) 

13. 8 ( 

1.2) 

55. 8( 

2.2) 

44.2( 

2.2) 

1.1) 

1.2( 

0.3) 

13. 3( 

1.1) 

49. 3( 

1.5) 

50.7  ( 

1.5) 

1.4) 

0.7( 

0.2) 

10. 2( 

1.0) 

49. 0( 

2.1) 

51. 0( 

2.1) 

1.3) 

1.8( 

0.5) 

15. 5 ( 

1.3) 

57. 6( 

1.7) 

42.4  ( 

1.7) 

1.0) 

1.9( 

0.3) 

19. 5 ( 

1.1) 

67. 4( 

1.6) 

32.6  ( 

1.6) 

1.3) 

3.K 

0.7) 

19. 4( 

1.6) 

60. 3( 

1.4) 

39. 7< 

1.4) 

1.1) 

1.3( 

0.3) 

12. 8 ( 

1.0) 

54. 1( 

1.6) 

45.9  ( 

1.6) 

1.1) 

3.0( 

0.5) 

25. 3( 

1.4) 

72. 4( 

1.3) 

27.6  ( 

1.3) 

1.0) 

1.5( 

0.3) 

19. 4( 

1.2) 

70. 1( 

1.4) 

29. 9( 

1.4) 

Guam 

Virgin  Islands 


1933 

905 


O.K  0.0)  [ 0\] 
O.K  0.0)  [ 0\] 


191. 1(  0.8) 
178. 0(  1.2) 


0.4(  0.1) 

0.0(  0.0) 


4.9(  0.5) 
0.4(  0.2) 


28. 0(  1.2) 

11. 8(  1.6) 


72. 0(  1.2) 

88. 2(  1.6) 


best  copy  available 


82-  Q 

ERIC 


Table  K-8 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  4 Weighted  Percentages  and  Composite  Scale  Means 
Percent  of  Students  At  or  Above  the  Achievement  Levels 
Revised  Results 


Alabama 
Arizona  . 

Arkansas 

California 

Colorado 

Connecticut 

Delaware 

District  of  Columbia 

Florida 

Georgia 

Hawaii 

Idaho 


N WEIGHTED  PCT  [CV] 


MEAN 


ADVANCED 


PRFCIENT 


BASIC 


< BASIC 


2605 

1.9( 

0.1) 

2741 

1.8( 

0.0) 

2621 

1.2( 

0.0) 

2412 

12. 3( 

0.3) 

2906 

1.7( 

0.1) 

2600 

1.2( 

0.0) 

3k] 

208. 3( 

1.6) 

2k] 

215. 3( 

1.1) 

4k] 

210. 2( 

0.9) 

3k] 

208. 4( 

1.6) 

3k] 

221.0 ( 

1.0) 

3k] 

226.8 ( 

1.1) 

0.5( 

0.1) 

10. 1( 

1.2) 

0.8( 

0.2) 

13. 1( 

0.9) 

0.4( 

0.2) 

9.7( 

0.7) 

1.3( 

0.4) 

12. 4( 

1.2) 

1.5( 

0.4) 

17. 5 ( 

1.0) 

2.9( 

0.5) 

24. 4( 

1.4) 

43. 0( 

2.1) 

57. 0( 

2.1) 

53. 5( 

1.6) 

46. 5( 

1.6) 

46. 9( 

1.5) 

53. 1( 

1.5) 

46. 4( 

1.9) 

53. 6( 

1.9) 

60. 8 ( 

1.4) 

39. 2( 

1.4) 

67.3  ( 

1.6) 

32. 7( 

1.6) 

2040 

0.3( 

0.0) 

[ 

Ok] 

217.9 ( 

0.8) 

1.8( 

2399 

0.2( 

0.0) 

[ 

Ok] 

192.6 ( 

0.5) 

0.9( 

2828 

5.4( 

0.2) 

t 

3k] 

213. 7( 

1.5) 

1.2( 

2766 

3.4( 

0.1) 

t 

4k] 

215. 6( 

1.2) 

l.K 

2625 

0.5( 

0.0) 

t 

2k] 

214.1 ( 

1.3) 

l.K 

2784 

0.6( 

0.0) 

t 

3k] 

221. 6 ( 

1.0) 

0.7( 

0.3) 

16. 5( 

0.9) 

54.6  ( 

1.0) 

45. 4( 

1.0) 

0.2) 

5.5( 

0.3) 

23. 1( 

0.9) 

76. 9( 

0.9) 

0.3) 

13. 3( 

1.4) 

51. 6( 

1.7) 

48. 4( 

1.7) 

0.3) 

15. 3( 

1.2) 

53. 1( 

1.7) 

46. 9( 

1.7) 

0.2) 

14. 6( 

0.9) 

51. 9( 

1.8) 

48. 1( 

1.8) 

0.3) 

15. 7( 

1.0) 

62. 7( 

1.7) 

37. 3( 

1.7) 

Indiana 

2593 

2.7( 

0.1) 

t 

3k] 

221. 0( 

Iowa 

2770 

1.3( 

0.0) 

t 

4k] 

229. 9 ( 

Kentucky 

2703  . 

1.8( 

0.1) 

t 

4k] 

215. 0( 

Louisiana 

2792 

2.2( 

0.1) 

t 

3k] 

204. 1( 

Maine 

1898 

0.6( 

0.0) 

t 

3k] 

231. 6( 

Maryland 

2844 

2.0( 

0.1) 

t 

3k] 

217.3 ( 

Massachusetts 

2549 

2.K 

0.1) 

t 

4k] 

226. 6( 

Michigan 

2412 

4.K 

0.1) 

t 

4k] 

219. 9( 

Minnesota 

2640 

2.K 

0.1) 

t 

4k] 

228. 5 ( 

Mississippi 

2712 

1.4( 

0.0) 

t 

3k] 

201. 8( 

Missouri 

2509 

2.K 

0.1) 

t 

5k] 

222. 2 ( 

Nebraska 

2327 

0.7( 

0.0) 

t 

3k] 

225. 3( 

New  Hampshire 

2265 

0.5( 

0.0) 

t 

3k] 

229. 7( 

New  Jersey 

2231 

2.8( 

0.1) 

t 

4k] 

227. 1( 

New  Mexico 

2342 

0.8( 

0.1) 

t 

6k] 

213.3 ( 

New  York 

2284 

6.7( 

0.2) 

t 

3k] 

218. 4 ( 

North  Carolina 

2884 

3.0( 

0.1) 

t 

3k] 

212. 9 ( 

North  Dakota 

2193 

0.3( 

0.0) 

t 

4k] 

228. 7( 

Ohio 

2637 

4.9( 

0.1) 

t 

3k] 

218. 7( 

Oklahoma 

2254 

1.6( 

0.0) 

t 

3k] 

220. 3( 

Pennsylvania 

2740 

4.6( 

0.2) 

t 

4k] 

224.3 ( 

Rhode  Island 

2390 

0.4( 

0.0) 

t 

5k] 

215. 4 ( 

South'  Carolina 

2771 

1.8( 

0.1) 

t 

3k] 

212. 5 ( 

Tennessee 

2708 

2.4( 

0.1) 

t 

3k] 

210.9 ( 

Texas 

2623 

9.0( 

0.3) 

t 

4k] 

217.9 ( 

Utah 

2799 

1.3( 

0.0) 

t 

2k] 

224. 0( 

Virginia 

2786 

2.8( 

0.1) 

t 

3k] 

220. 8( 

West  Virginia 

2786 

0.9( 

0.0) 

t 

4k] 

215. 3( 

Wisconsin 

2780 

2.K 

0.1) 

t 

4k] 

228. 7( 

Wyoming 

2605 

0.3( 

0.0) 

t 

3k] 

225. 4( 

Guam 

1933 

O.K 

0.0) 

t 

Ok] 

192. 8( 

Virgin  Islands 

905 

O.K 

0.0) 

t 

Ok] 

178.9 ( 

1.0) 

1.0( 

0.2) 

15. 6( 

1.1) 

59. 8( 

1.7) 

40. 2( 

1.7) 

1.0) 

2.3  ( 

0.4) 

26. 0( 

1.2) 

72. 4( 

1.5) 

27. 6( 

1.5) 

1.0) 

1.0( 

0.3) 

12. 6( 

1.2) 

50. 9( 

1.5) 

49. 1( 

1.5) 

1.5) 

0.4( 

0.2) 

7.6( 

0.8) 

38. 8( 

2.0) 

61. 2( 

2.0) 

1.0) 

2.4( 

0.5) 

27. 4 ( 

1.5) 

74. 8( 

1.5) 

25. 2( 

1.5) 

1.3) 

2.2( 

0.3) 

18. 4( 

1.2) 

55. 1( 

1.6) 

44. 9( 

1.6) 

1.2) 

2.4( 

0.5) 

23. 3( 

1.5) 

68. 5( 

1.6) 

31. 5( 

1.6) 

1.7) 

1.2( 

0.4) 

18. 5 ( 

1.7) 

60. 5( 

2.2) 

39. 5( 

2.2) 

0.9) 

2.5( 

0.4) 

25. 9( 

1.3) 

70. 6( 

1.6) 

29. 4( 

1.6) 

1.1) 

0.3( 

0.1) 

6.3( 

0.6) 

35. 8( 

1.3) 

64. 2( 

1.3) 

1.2) 

1.3( 

0.3) 

18. 6 ( 

1.3) 

62. 1( 

1.7) 

37. 9( 

1.7) 

1.2) 

2.0( 

0.5) 

21. 8 ( 

1.6) 

66. 6( 

1.8) 

33. 4( 

1.8) 

1.2) 

2.K 

0.4) 

24. 9( 

1.6) 

72.3  ( 

1.6) 

27. 7( 

1.6) 

1.5) 

2.5( 

0.6) 

24. 6 ( 

1.5) 

68. 2( 

2.1) 

31. 8( 

2.1) 

1.4) 

0.6( 

0.2) 

11. 1( 

1.3) 

49.8  ( 

2.0) 

50. 2( 

2.0) 

1.2) 

1.5( 

0.3) 

17. 0( 

1.3) 

57.0  ( 

1.8) 

43. 0( 

1.8) 

1.1) 

1.2( 

0.3) 

12.7  ( 

0.8) 

50. 3( 

1.6) 

49.7  ( 

1.6) 

0.8) 

1.3( 

0.3) 

22.2  ( 

1.1) 

72. 5( 

1.3) 

27.5  < 

1.3) 

1.2) 

1.4( 

0.3) 

16. 1( 

1.2) 

57. 0( 

1.7) 

43. 0( 

1.7) 

1.0) 

0.8( 

0.3) 

14. 0( 

1.2) 

59. 5( 

1.7) 

40. 5( 

1.7) 

1.3) 

2.0( 

0.4) 

21. 8 ( 

1.5) 

64. 9( 

2.0) 

35. 1( 

2.0) 

1.5) 

1.2( 

0.4) 

13.3  ( 

1.1) 

54. 2( 

2.2) 

45. 8( 

2.2) 

1.1) 

0.9( 

0.3) 

. 12. 8 ( 

1.1) 

47.6  ( 

1.7) 

52. 4( 

1.7) 

1.4) 

0.5( 

0.2) 

9.9( 

1.0) 

47.3  ( 

2.0) 

52. 7( 

2.0) 

1.2) 

1.2( 

0.3) 

15. 0( 

1.2) 

55. 7( 

1.6) 

44. 3( 

1.6) 

1.0) 

1.4( 

0.3) 

19. 0( 

1.1) 

65. 6( 

1.7) 

34. 4( 

1.7) 

1.3) 

2.3( 

0.5) 

18.8  ( 

1.5) 

58. 6( 

1.4) 

41. 4( 

1.4) 

1.1) 

1.0( 

0.3) 

12.2  ( 

0.9) 

52. 4( 

1.5) 

47. 6 ( 

1.5) 

1.1) 

2.2( 

0.4) 

24. 5( 

1.4) 

70. 9( 

1.4) 

29. 1( 

1.4) 

0.9) 

1.0( 

0.3) 

18. 7( 

1.1) 

68. 6( 

1.4) 

31. 4( 

1.4) 

0.8) 

0.3( 

0.2) 

4.7  ( 

0.5) 

26. 3( 

1.4) 

73. 7< 

1.4) 

1.2) 

0.0( 

0.0) 

0.5( 

0.2) 

10. 6( 

1.4) 

89. 4( 

1.4) 

best  copy  available 
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Table  K-9 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  8 Weighted  Percentages  and  Composite  Scale  Means 
Percent  of  Students  At  or  Above  the  Achievement  Levels 
Original  Results 


N WEIGHTED  PCT  [CV]  MEAN 


1992 

2522 

2.0( 

0.1) 

[ 

3k] 

251. 3( 

1.7) 

1990 

2531 

2.2( 

0.1) 

[ 

3k] 

252. 9( 

1.1) 

1992 

2617 

1.7( 

0.1) 

[ 

4k] 

264. 6 ( 

1.3)> 

1990 

2558 

1.8( 

0.1) 

[ 

3k] 

259. 6( 

1.3) 

1992 

2556 

1.2( 

0.0) 

[ 

2k] 

255. 4( 

1.2) 

1990 

2669 

1.5( 

0.0) 

[ 

2k] 

256. 2( 

0.9) 

1992 

2516 

12. 5 ( 

0.3) 

[ 

3k] 

260. 1( 

1.7) 

1990 

2424 

14. 2 ( 

0.4) 

[ 

3k] 

256. 3( 

1.3) 

1992 

2799 

1.6( 

0.0) 

[ 

2k] 

271. 7( 

1.1)> 

1990 

2675 

1.7( 

0.0) 

[ 

2k] 

267. 4( 

0.9) 

1992 

2613 

1.2( 

0.0) 

[ 

3k] 

273. 1( 

1.1)> 

1990 

2672 

1.4( 

0.0) 

[ 

3k] 

269. 9( 

1.0) 

1992 

1934 

0.3( 

0.0) 

[ 

Ok] 

262. 1( 

1.0) 

1990 

2110 

0.3( 

0.0) 

[ 

Ok] 

260. 7( 

0.9) 

1992 

1816 

0.2( 

0.0) 

[ 

Ok] 

233. 9( 

0.9)> 

1990 

2135 

0.2( 

0.0) 

[ 

Ok] 

231. 4( 

0.9) 

1992 

2549 

4.6( 

0.2) 

[ 

3k] 

259. 1( 

1.5) 

1990 

2534 

5.5( 

0.2) 

[ 

3k] 

255. 3( 

1.2) 

1992 

2589 

3.K 

0.1) 

[ 

4k] 

258. 5( 

1.2) 

1990 

2766 

3.7  ( 

0.1) 

[ 

4k] 

258. 8( 

1.3) 

1992 

2454 

0.5( 

0.0) 

[ 

Ok] 

256. 6( 

0.9)> 

1990 

2551 

0.5( 

0.0) 

[ 

Ok] 

251. 0( 

0.8) 

1992 

2615 

0.7  ( 

0.0) 

[ 

2k] 

274. 4( 

0.8)> 

1990 

2716 

0.8( 

0.0) 

[ 

Ik] 

271. 4( 

0.8) 

1992 

2659 

3.0( 

0.1) 

[ 

3k] 

269. 4( 

1.2) 

1990 

2569 

3.2( 

0.1) 

[ 

3k] 

267. 3( 

1.2) 

1992 

2816 

1.4( 

0.1) 

[ 

4k] 

282. 8( 

1.0)> 

1990 

2474 

1.5( 

0.1) 

[ 

4k] 

278. 0( 

1.1) 

1992 

27  56 

1.8( 

0.1) 

[ 

3k] 

261. 4( 

1.1)> 

1990 

2680 

2.K 

0.1) 

[ 

4k] 

257. 1( 

1.2) 

1992 

2582 

1.9( 

0.1) 

[ 

4k] 

249. 1( 

1.7) 

1990 

2572 

2.2( 

0.1) 

[ 

4k] 

246. 4( 

1.2) 

1992 

2464 

0.6( 

0.0) 

[ 

2k] 

278. 0( 

1.0) 

1990 

0 

0.0( 

0.0) 

[ 

Ok] 

*•***( 

0.0) 

1992 

2399 

1.9( 

0.1) 

[ 

3k] 

264. 2( 

1.3) 

1990 

2794 

2.2( 

0.0) 

[ 

2k] 

260. 8( 

1.4) 

1992 

2456 

2.K 

0.1) 

[ 

4k] 

272. 1( 

1.1) 

1990 

0 

0.0( 

0.0) 

[ 

Ok] 

**•**( 

0.0) 

1992 

2616 

4.2( 

0.1) 

[ 

2k] 

266. 6( 

1.4) 

1990 

2587 

4.8( 

0.1) 

[ 

3k] 

264. 4( 

1.2) 

1992 

2471 

2.0( 

0.1) 

[ 

2k] 

281. 8( 

1.0)> 

1990 

2584 

2.4( 

0.1) 

[ 

4k] 

275. 4( 

0.9) 

1992 

2498 

1.4( 

0.0) 

[ 

3k] 

245. 5( 

1.2) 

1990 

0 

0.0( 

0.0) 

[ 

Ok] 

****•( 

0.0) 

1992 

2666 

2.2( 

0.0) 

[ 

2k] 

270. 4( 

1.2) 

1990 

0 

0.0( 

0.0) 

[ 

Ok] 

****•( 

0.0) 

ADVANCED 

PRFCIENT 

BASIC 

< BASIC 

l.K 

0.3) 

12.3  ( 

1.1) 

44. 3( 

2.0) 

55. 7( 

2.0) 

l.K 

0.2) 

11.7  ( 

0.8) 

47. 2 ( 

1.6) 

52. 8( 

1.6) 

1.8( 

0.4) 

18. 8( 

1.4) 

61. 4( 

1.8)> 

38. 6( 

1.8)< 

1.5( 

0.4) 

16. 1( 

1.1) 

54. 8( 

1.8) 

45. 2( 

1.8) 

1.0( 

0.3) 

12.9  ( 

1.0) 

50. 2( 

1.7) 

49. 8( 

1.7) 

0.9( 

0.2) 

12. 4( 

1.0) 

51. 2( 

1.3) 

48. 8( 

1.3) 

2.6( 

0.7) 

19.7  ( 

1.4) 

55. 1( 

2.0) 

44.9  ( 

2.0) 

2.0( 

0.4) 

15. 9( 

1.3) 

50. 9( 

1.6) 

49. 1( 

1.6) 

2.5( 

0.5) 

26. 2( 

1.3)> 

69. 2( 

1.3)> 

30. 8( 

1.3)< 

2.2( 

0.4) 

21. 6( 

1.0) 

64.3  ( 

1.1) 

35.7  ( 

1.1) 

4.K 

0.6) 

30. 1( 

1.1)> 

68. 9( 

1.4) 

31. 1< 

1.4) 

3.9( 

0.4) 

26. 2( 

1.1) 

65. 9( 

1.3) 

34. 1( 

1.3) 

2.5( 

0.4) 

18. 5( 

1.1) 

57. 0( 

1.2) 

43. 0( 

1.2) 

2.K 

0.5) 

18.6  ( 

0.9) 

54.5  ( 

1.3) 

45. 5( 

1.3) 

0.7  ( 

0.2) 

5.6( 

1.0) 

26. 0( 

1.3)> 

74. 0( 

1.3)< 

0.8( 

0.2) 

3.8( 

0.7) 

21. 1( 

1.0) 

78. 9( 

1.0) 

1.8( 

0.4) 

17.6  ( 

1.3) 

54.7  ( 

1.9) 

45. 3( 

1.9) 

1.7  ( 

0.4) 

14.8  ( 

1.0) 

49. 2( 

1.4) 

50. 8( 

1.4) 

1.4( 

0.3) 

15. 9( 

1.0) 

53. 5( 

1.5) 

46. 5( 

1.5) 

2.6( 

0.5) 

17.2  ( 

1.3) 

53. 4( 

1.5) 

46. 6( 

1.5) 

2.2( 

0.4) 

16. 5( 

0.8) 

51. 2( 

1.2)> 

48. 8( 

1.2)< 

1.8( 

0.3) 

14.3  ( 

0.8) 

45. 3( 

1.0) 

54. 7( 

1.0) 

2.5( 

0.4) 

26. 9( 

1.2) 

73. 4( 

1.1) 

26. 6 ( 

1.1) 

1.5( 

0.4) 

23. 0( 

1.4) 

70. 1( 

1.2) 

29. 9 ( 

1.2) 

3.0( 

0.4) 

23. 8 ( 

1.3) 

65. 8( 

1.5) 

34. 2( 

1.5) 

2.9( 

0.6) 

20. 6( 

1.2) 

63. 2( 

1.6) 

36. 8( 

1.6) 

4.7  ( 

0.7) 

37. 2 ( 

1.4)> 

81. 3( 

1.2)> 

18.7  ( 

1.2)< 

3.8( 

0.5) 

30. 4( 

1.5) 

76. 3( 

1.1) 

23.7  ( 

1.1) 

1.9( 

0.4) 

16.7  ( 

1.1) 

57.3  ( 

1.3)> 

42. 7( 

1.3X 

1.2( 

0.2) 

13. 5 ( 

0.9) 

50. 5( 

1.8) 

49. 5( 

1.8) 

0.5( 

0.2) 

9.6( 

1.2) 

42. 4( 

2.0) 

57. 6 ( 

2.0) 

0.6( 

0.2) 

7.6( 

1.0) 

38. 6( 

1.7) 

61. 4( 

1.7) 

3.7  ( 

0.6) 

30. 8( 

1.9) 

77. 5( 

1.3) 

22. 5( 

1.3) 

0.0) 

****•( 

0.0) 

****•( 

0.0) 

0.0) 

3.7  ( 

0.6) 

23. 8( 

1.3) 

58. 7( 

1.5) 

41. 3( 

1.5) 

3.K 

0.6) 

19. 9( 

1.2) 

55. 8( 

1.7) 

44. 2( 

1.7) 

3.3( 

0.5) 

27. 9 ( 

1.4) 

67. 9 ( 

1.5) 

32. 1( 

1.5) 

0.0) 

***•*( 

0.0) 

0.0) 

0.0) 

2.6( 

0.5) 

23. 1( 

1.7) 

63. 2( 

1.6) 

36. 8( 

1.6) 

2.4( 

0.4) 

19.7  ( 

1.4) 

60. 3( 

1.4) 

39.7  ( 

1.4) 

5.6( 

0.7)> 

36.7  ( 

1.2)> 

78. 5( 

1.2)> 

21. 5( 

1.2)< 

3.7( 

0.4) 

28.7  ( 

1.2) 

73. 9( 

1.3) 

26. 1( 

1.3) 

0.5( 

0.2) 

8.5( 

0.8) 

38. 3( 

1.5) 

61. 7( 

1.5) 

0.0) 

*•***( 

0.0) 

0.0) 

****•( 

0.0) 

2.7  ( 

0.4) 

23.7  ( 

1.3) 

67. 9( 

1.6) 

32. 1( 

1.6) 

0.0) 

***•*( 

0.0) 

0.0) 

0.0) 

SIGNIFICANT  INCREASE  (OR  DECREASE  "<")  BETWEEN  1990  AND  1992 
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Table  K-9  (continued) 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  8 Weighted  Percentages  and  Composite  Scale  Means 
Percent  of  Students  At  or  Above  the  Achievement  Levels 
Original  Results 


Nebraska 

1992 

N 

2285 

WEIGHTED  PCT 
0.8(  0.0) 

[CV] 

[ 3M 

MEAN 

277. 0(  1.1) 

ADVANCED 
3.6(  0.5) 

PRFCIENT 
31. 7(  1.9) 

BASIC 
75. 4(  1.2) 

< BASIC 
24. 6(  1.2) 

1990 

2519. 

0.9( 

0.0) 

275.7 ( 

1.0) 

3.6( 

0.6) 

29. 9( 

1.4) 

74. 3( 

1.1) 

25. 7( 

1.1) 

New  Hampshire 

1992 

2535 

0.5( 

0.0) 

2\] 

277. 6( 

1.0)> 

3.5( 

0.6) 

30. 0( 

1.5)> 

76. 8( 

1.0)> 

23. 2 ( 

1.0)< 

1990 

2568 

0.5( 

0.0) 

273. K 

0.9) 

3.2( 

0.5) 

25. 0( 

1.2) 

71. 3( 

1.6) 

28.7  ( 

1.6) 

New  Kersey 

1992 

2174 

3.2( 

0.1) 

4\] 

271. 2( 

1.6) 

3.9( 

0.6) 

27. 6 ( 

1.4) 

67  .3( 

1.8) 

32. 7( 

1.8) 

1990 

2710 

3.5( 

0.1) 

4\] 

269.7 ( 

1.1) 

3.9( 

0.5) 

25.3  ( 

1.3) 

64. 5( 

1.6) 

35. 5( 

1.6) 

New  Mexico 

1992 

2561 

0.8( 

0.0) 

IM 

258. 8( 

0.9)> 

l.K 

0.3) 

13.8  ( 

1.0) 

54. K 

1.4) 

45. 9( 

1.4) 

1990 

2643 

0.8( 

0.0) 

IM 

256. 4( 

0.7) 

1.2( 

0.3) 

12.8  ( 

0.9) 

50.7  ( 

1.3) 

49. 3( 

1.3) 

New  York 

1992 

2158 

6.5( 

0.2) 

3\] 

265.7 ( 

2.1) 

3.7  ( 

0.6) 

23.8  ( 

1.6)> 

62. 5 ( 

2.3) 

37  .5( 

2.3) 

1990 

2302 

7.7  ( 

0.2) 

2\] 

260. 8( 

1.4) 

3.K 

0.5) 

18. 8 ( 

1.0) 

56. 5( 

1.7) 

43. 5 ( 

1.7) 

North  Carolina 

1992 

2769 

3.2( 

0.1) 

3M 

257. 6( 

1.2)> 

1.5( 

0.3) 

15. 2( 

1.0)> 

52. 8( 

1.5)> 

47. 2( 

1.5)< 

1990 

2843 

3.6( 

0.1) 

2\] 

250. 3( 

1.1) 

0.8( 

0.4) 

11. 3( 

0.8) 

44. 4( 

1.4) 

55. 6( 

1.4) 

North  Dakota 

1992 

2314 

0.3( 

0.0) 

3\] 

282. 6( 

1.2) 

3.8( 

0.6) 

35.7  ( 

1.7) 

82. 4( 

1.3) 

17. 6 { 

1.3) 

1990 

2485 

0.4  ( 

0.0) 

4M 

281. K 

1.2) 

3.9( 

0.6) 

33.8  ( 

2.0) 

81. 0( 

1.6) 

19. 0( 

1.6) 

Ohio 

1992 

2535 

5.4( 

0.2) 

4\] 

267. 4( 

1.5) 

2.4( 

0.5) 

22.4  ( 

1.4) 

64. 3( 

2.0) 

35. 7( 

2.0) 

1990 

2673 

5.6( 

0.1) 

2\] 

264. 0( 

1.0) 

2.0( 

0.3) 

18. 5( 

1.2) 

59. 6 ( 

1.4) 

40. 4( 

1.4) 

Oklahoma 

1992 

2141 

1.5( 

0.0) 

3\] 

267. 4( 

1.2)> 

1.5( 

0.3) 

21. K 

1.2)> 

65. 2( 

2.0) 

34. 8( 

2.0) 

1990 

2222 

1.8( 

0.0) 

2\] 

263. 2( 

1.3) 

1.6( 

0.5) 

16. 9( 

1.3) 

59. 2( 

1.6) 

40. 8( 

1.6) 

Pennsylvania 

1992 

2612 

4.5( 

0.1) 

3\] 

270.7 ( 

1.5) 

3.2( 

0.7) 

25. 9( 

1.5) 

67. 2( 

1.7) 

32. 8( 

1.7) 

1990 

2528 

5.4  ( 

0.2) 

3%] 

266. 4( 

1.6) 

2.3( 

0.4) 

21. 4( 

1.5) 

62. 9( 

2.0) 

37  .K 

2.0) 

Rhode  Island 

1992 

2120 

0.4( 

0.0) 

265. K 

0.7)> 

1.8( 

0.3) 

19. 7( 

1.3) 

62. 0( 

1.2)> 

38. 0( 

1.2X 

1990 

2675 

0.4( 

0.0) 

2\] 

260. 0( 

0.6) 

1.8( 

0.3) 

18. 3( 

1.0) 

54. 9( 

0.9) 

45. K 

0.9) 

South  Carolina 

1992 

2625 

1.8( 

0.0) 

2\] 

260. 0( 

1.0) 

2.K 

0.5) 

18. 0( 

1.1) 

53. 4( 

1.2) 

46. 6( 

1.2) 

1990 

0 

0.0( 

0.0) 

0\] 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

*****( 

0.0) 

****«( 

0.0) 

Tennessee 

1992 

2485 

2.3( 

0.1) 

3%] 

258. 0( 

1.4) 

1.2( 

0.4) 

14. 7( 

1.2) 

52. 6( 

1.8) 

47.4  ( 

1.8) 

1990 

0 

0.0( 

0.0) 

0%] 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

****•( 

0.0) 

****«( 

0.0) 

Texas 

1992 

2614 

8.8( 

0.3) 

263. 8( 

1.3)>. 

3.6( 

0.6) 

21. 2( 

1.4)> 

57. 8( 

1.5)> 

42. 2( 

1.5)< 

1990 

2542 

10. 2( 

0.3) 

3%] 

258. 2( 

1.4) 

2.0( 

0.4) 

15. 9( 

1.0) 

52. 0( 

1.7) 

48. 0( 

1.7) 

Utah 

1992 

2726 

1 . 2 ( 

0.0) 

3%] 

273. 6( 

0.7) 

2.7  ( 

0.5) 

27. 2( 

1.1) 

72. K 

1.3) 

27. 9 ( 

1.3) 

1990 

0 

0.0( 

0.0) 

0%] 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

*****( 

0.0) 

«****( 

0.0) 

Virginia 

1992 

2710 

2.8( 

0.1) 

4%] 

267. K 

1.2) 

3.2( 

0.5) 

22. 9( 

1.2) 

62. 3( 

1.6) 

37.7  { 

1.6) 

1990 

2661 

3.0( 

0.1) 

3%] 

264. 3( 

1.5) 

4.K 

0.8) 

20.7  ( 

1.6) 

57.7  ( 

1.6) 

42.3  ( 

1.6) 

West  Virginia 

1992 

2690 

0.9( 

0.0) 

3%] 

258. 2( 

1.0) 

0.7  ( 

0.2) 

12. 6( 

0.9) 

52. 9( 

1.5) 

47. K 

1.5) 

1990 

2600 

l.K 

0.0) 

3%] 

255. 9( 

1.0) 

l.K 

0.2) 

12. K 

0.9) 

49. 3( 

1.2) 

50.7  < 

1.2) 

Wisconsin 

1992 

2814 

2.3( 

0.1) 

6%] 

277  .3( 

1.5) 

3.9( 

0.6) 

32. K 

1.4) 

75. 6( 

1.9) 

24.4  ( 

1.9) 

1990 

2750 

2.4( 

0.1) 

3%] 

274. 5( 

1.3) 

3.6( 

0.5) 

28. 8( 

1.5) 

72. 3( 

1.7) 

27.7  { 

1.7) 

Wyoming 

1992 

2444 

0.3( 

0.0) 

2%] 

274. 4( 

0.9)> 

2.4( 

0.5) 

25. 8( 

1.0) 

73. 0( 

1.3) 

27.0  ( 

1.3) 

1990 

2701 

0.3( 

0.0) 

1%] 

272. K 

0.7) 

2.0( 

0.3) 

23. 6( 

1.0) 

71. K 

1.3) 

28. 9( 

1.3) 

Gueun 

1992 

1496 

O.K 

0.0) 

1%] 

234. 3( 

1.0)> 

0.6( 

0.2) 

7.0( 

0.7) 

29. 5 ( 

1.4) 

70. 5< 

1.4) 

1990 

1617 

O.K 

0.0) 

0%] 

231. 8( 

0.7) 

0.5( 

0.2) 

5.2( 

0.6) 

26.7  ( 

1.0) 

73. 3< 

1.0) 

Virgin  Islands 

1992 

1479 

O.K 

0.0) 

0%] 

221. 8( 

1.1)> 

0.0( 

0.1) 

0.9( 

0.3) 

12.7  ( 

1.0) 

87. 3( 

1.0) 

1990 

1326 

O.K 

0.0) 

0%] 

218. 7( 

0.9) 

O.K 

6.1) 

0.9( 

0.4) 

10. 4( 

1.1) 

89.6  ( 

1.1) 

> INDICATES  A SIGNIFICANT  INCREASE  (OR  DECREASE  "<")  BETWEEN  1990  AND  1992 
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Table  K-10 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  8 Weighted  Percentages  and  Composite  Scale  Means 
Percent  of  Students  At  or  Above  the  Achievement  Levels 
Revised  Results 


N 

WEIGHTED  PCT 

ICV) 

MEAN 

ADVANCED 

PRFCIENT 

BASIC 

< BASIC 

Alabama 

1992 

2522 

2.0( 

0.1) 

3%} 

252.2 ( 

1.7) 

0.9( 

0.3) 

10. 2( 

0.9) 

38. 8( 

1.9) 

61. 2( 

1.9) 

1990 

2531 

2.2( 

0.1) 

3%} 

252. 9( 

1.1) 

1.0( 

0.2) 

9.K 

0.7) 

40. 3( 

1.7) 

59. 7( 

1.7) 

Arizona 

1992 

. 2617 

1.7( 

0.1) 

4\) 

265. 4( 

1.3)> 

1.4( 

0.3) 

15. 1( 

1.3) 

54. 7( 

1.8)> 

45. 3( 

1.8)< 

1990 

2558 

1.8( 

0.1) 

3%} 

259. 6 ( 

1.3) 

1.3( 

0.4) 

12.7  ( 

0.9) 

47.5  ( 

1.8) 

52. 5 ( 

1.8) 

Arkansas 

1992 

2556 

1.2( 

0.0) 

2\} 

256.3 ( 

1.2) 

0.7( 

0.2) 

10. 0( 

0.8) 

44. 3( 

1.8) 

55. 7( 

1.8) 

1990 

2669 

1.5( 

0.0) 

2\} 

256. 2( 

0.9) 

0.7( 

0.2) 

9.3( 

0.7) 

43. 9( 

1.2) 

56. 1( 

1.2) 

California 

1992 

2516 

12. 5 ( 

0.3) 

3\} 

260. 9( 

1.7)> 

2.2( 

0.7) 

16. 2( 

1.3) 

50. 4( 

1.9) 

49. 6( 

1.9) 

1990 

2424 

14. 2 ( 

0.4) 

3\} 

256. 3( 

1.3) 

1.7( 

0.3) 

12.5  ( 

1.1) 

44. 6( 

1.7) 

55. 4( 

1.7) 

Colorado 

1992 

2799 

1.6( 

0.0) 

2\} 

272.4 ( 

1.0)> 

2.K 

0.4) 

21. 6( 

1.2)> 

63. 9( 

1.4)> 

36. 1( 

1.4)< 

1990 

2675 

1.7( 

0.0) 

2\) 

267. 4 ( 

0.9) 

1.9( 

0.4) 

16. 9( 

1.0) 

57.5  ( 

1.2) 

42. 5( 

1.2) 

Connecticut 

1992 

2613 

1.2( 

0.0) 

3\} 

273. 7 ( 

1.1)> 

3.2( 

0.6) 

25. 7( 

1.1)> 

64. 4( 

1.4) 

35. 6( 

1.4) 

1990 

2672 

1.4( 

0.0) 

3\) 

269. 9( 

1.0) 

3.4( 

0.4) 

21. 7( 

0.9) 

59. 9 ( 

1.4) 

40. 1( 

1.4) 

Delaware 

1992 

1934 

0.3( 

0.0) 

0\} 

262. 9( 

1.0) 

2.3( 

0.4) 

15. 1( 

1.0) 

51. 6( 

1.2) 

48. 4( 

1.2) 

1990 

2110 

0.3( 

0.0) 

0\) 

260. 7( 

0.9) 

1.9( 

0.4) 

14. 2 ( 

0.8) 

47.8  ( 

1.5) 

52. 2( 

1.5) 

Diet  of  Columbia 

1992 

1816 

0.2( 

0.0) 

0\} 

234.9 ( 

0.9)> 

0.6( 

0.2) 

4.4( 

0.9) 

21. 8( 

1.1)> 

78. 2( 

1.1)< 

1990 

2135 

0.2( 

0.0) 

Ok) 

231. 4( 

0.9) 

0.8( 

0.2) 

3.K 

0.6) 

16. 6( 

1.0) 

83. 4( 

1.0) 

Florida 

1992 

2549 

4.6( 

0.2) 

3\) 

259. 9( 

1.5)> 

1.5( 

0.3) 

14. 6 ( 

1.2) 

48. 8( 

1.9)> 

51. 2( 

1.9)< 

1990 

2534 

5.5( 

0.2) 

3\) 

255.3 ( 

1.2) 

1.4( 

0.3) 

12. 0( 

0.9) 

42. 7( 

1.4) 

57.3  ( 

1.4) 

Georgia 

1992 

2589 

3.K 

0.1) 

4k) 

259. 4( 

1.2) 

l.K 

0.3)< 

12. 7( 

0.9) 

47.9  ( 

1.7) 

52. 1( 

1.7) 

1990 

2766 

3.7( 

0.1) 

4k) 

258. 8 ( 

1.3) 

2.4( 

0.4) 

13.8  ( 

1.2) 

47. 2( 

1.5) 

52. 8( 

1.5) 

Hawaii 

1992 

2454 

0.5( 

0.0) 

Ok) 

257. 4( 

0.9)> 

1.8( 

0.3) 

13. 5( 

0.7) 

46. 2( 

1.1)> 

53. 8( 

1.1)< 

1990 

2551 

0.5( 

0.0) 

Ok) 

251. 0( 

0.8) 

1.6( 

0.3) 

11. 7( 

0.7) 

40. 0( 

1.0) 

60. 0( 

1.0) 

Idaho 

1992 

2615 

0.7( 

0.0) 

2k) 

275. 1( 

0.7)> 

2.K 

0.3) 

21. 7( 

1.2) 

67.9  ( 

1.0)> 

32. 1( 

1.0)< 

1990 

2716 

0.8( 

0.0) 

Ik) 

271.4 ( 

0.8) 

1.2( 

0.3) 

17.9  ( 

1.1) 

63. 4 ( 

1.2) 

36. 6( 

1.2) 

Indiana 

1992 

2659 

3.0( 

0.1) 

3k) 

270. 1( 

1.1) 

2.6( 

0.4) 

19. 7( 

1.2) 

59. 9( 

1.5) 

40. 1( 

1.5) 

1990 

2569 

3.2( 

0.1) 

3k) 

267. 3( 

1.2) 

2.5( 

0.5) 

16. 6( 

1.1) 

56. 4( 

1.5) 

43. 6 ( 

1.5) 

Iowa 

1992 

2816 

1.4( 

0.1) 

4k) 

283. 4( 

1.0)> 

3.9( 

0.7) 

31. 2( 

1.3)> 

76. 4( 

1.3)> 

23. 6 ( 

1.3)< 

1990 

2474 

1.5( 

0.1) 

4k) 

278. 0( 

1.1) 

3.3( 

0.5) 

25. 1( 

1.4) 

70. 0( 

1.2) 

30. 0( 

1.2) 

Kentucky 

1992 

2756 

1.8( 

0.1) 

3k) 

262. 2( 

1.1)> 

1.6( 

0.3) 

13. 8 ( 

1.1)> 

51. 2( 

1.5)> 

48. 8( 

1.5X 

1990 

2680 

2.K 

0.1) 

4k) 

257. 1( 

1.2) 

l.K 

0.3) 

10. 5( 

0.8) 

43. 0( 

1.7) 

57. 0( 

1.7) 

Louisiana 

1992 

2582 

1.9( 

0.1) 

4k) 

250. 0( 

1.7) 

0.5( 

0.2) 

7.2( 

1.0) 

36. 6( 

1.9) 

63. 4( 

1.9) 

1990 

2572 

2.2( 

0.1) 

4k) 

246. 4( 

1.2) 

0.5( 

0.2) 

5.4( 

0.6) 

31. 7( 

1.6) 

68. 3( 

1.6) 

Maine 

1992 

2464 

0.6( 

0.0) 

2k) 

278. 6( 

1.0) 

3.K 

0.6) 

25. 5( 

1.5) 

71. 6( 

1.3) 

28. 4( 

1.3) 

1990 

0 

0.0( 

0.0) 

Ok) 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

0.0) 

0.0) 

Maryland 

1992 

2399 

1.9( 

0.1) 

3k) 

264. 8( 

1.3)> 

3.2( 

0.5) 

19. 9( 

1.2) 

53. 9( 

1.4) 

46. 1( 

1.4) 

1990 

2794 

2.2( 

0.0) 

2k) 

260. 8( 

1.4) 

2.6( 

0.5) 

16. 6( 

1.2) 

49. 7( 

1.6) 

50.3  ( 

1.6) 

Massachusetts 

1992 

2456 

2.K 

0.1) 

4k) 

272. 8 ( 

1.0) 

2.8( 

0.5) 

23. 3( 

1.3) 

62. 8( 

1.5) 

37. 2( 

1.5) 

1990 

0 

0.0( 

0.0) 

Ok) 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

0.0) 

0.0) 

Michigan 

1992 

2616 

4.2( 

0.1) 

2k) 

267. 4( 

1.4) 

2.2( 

0.4) 

18.9  ( 

1.5) 

57.9  ( 

1.7) 

42. 1( 

1.7) 

1990 

2587 

4.8( 

0.1) 

3k) 

264. 4( 

1.2) 

2.K 

0.4) 

15.8  ( 

1.2) 

53. 3( 

1.7) 

46. 7( 

1.7) 

Minnesota 

1992 

2471 

2.0( 

0.1) 

2k) 

282. 4( 

1.0)> 

4.7( 

0.6) 

31. 1( 

1.2)> 

74. 2( 

1.3)> 

25. 8( 

1.3X 

1990 

2584 

2.4( 

0.1) 

4k) 

275. 4( 

0.9) 

3.3( 

0.5) 

23.3  ( 

1.2) 

67.5  ( 

1.1) 

32. 5( 

1.1) 

Mississippi 

1992 

2498 

1.4( 

0.0) 

3k) 

246. 5( 

1.2) 

0.3( 

0.1) 

6.4( 

0.7) 

33. 4( 

1.6) 

66. 6( 

1.6) 

1990 

0 

0.0( 

0.0) 

Ok) 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

0.0) 

0.0) 

Missouri 

1992 

2666 

2.2( 

0.0) 

2k) 

271. 1( 

1.2) 

2.3( 

0.4) 

19.5  ( 

1.2) 

62. 5( 

1.6) 

37. 5( 

1.6) 

1990 

0 

0.0( 

0.0) 

Ok) 

*****( 

0.0) 

0.0) 

*****( 

0.0) 

0.0) 

0.0) 

> INDICATES  A SIGNIFICANT 

INCREASE 

(OR  DECREASE 

■ 

■<■) 

BETWEEN  1990  AND 

1992 

(continued) 
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Table  K-10  (continued) 

NAEP  1992  Trial  State  Assessment  in  Mathematics 
Grade  8 Weighted  Percentages  and  Composite  Scale  Means 
Percent  of  Students  At  or  Above  the  Achievement  Levels 
Revised  Results 


WEIGHTED  PCT  [CV] 


PRFCIENT 


< BASIC 


Nebraska 

1992 

2285 

0.8( 

0.0) 

277.7 ( 

1.1) 

2.9( 

0.5) 

26.3  ( 

1.6) 

70. 2( 

1.3) 

29. 8( 

1.3) 

1990 

2519 

0.9( 

0.0) 

2%] 

275.7 ( 

1.0) 

3.3( 

0.5) 

24. 4( 

1.2) 

68. 2( 

1.3) 

31. 8( 

1.3) 

New  Hampshire 

1992 

2535 

0.5( 

0.0) 

2%} 

278. 2( 

1.0)> 

2.8( 

0.5) 

24. 9( 

1.4)> 

71. 2( 

1.3)> 

28. 8( 

1.3)< 

1990 

2568 

0.5( 

0.0) 

1%} 

273. K 

0.9) 

2.7  ( 

0.5) 

20.3  ( 

1.2) 

64.7  ( 

1.5) 

35. 3( 

1.5) 

New  Jersey 

1992 

2174 

3.2( 

0.1) 

4%] 

271. 9( 

1.6) 

3.2( 

0.4) 

23.7  ( 

1.3) 

62. 5 ( 

1.9) 

37. 5( 

1.9) 

1990 

2710 

3.5( 

0.1) 

4%] 

269.7 ( 

1.1) 

3.4( 

0.5) 

21. 3( 

1.1) 

58. 2( 

1.5) 

41. 8( 

1.5) 

New  Mescico 

1992 

2561 

0.8( 

0.0) 

1%] 

259. 6( 

0.9)> 

0.9( 

0.3) 

10. 9( 

0.8) 

47. 6( 

1.3)> 

52. 4( 

1.3)< 

1990 

2643 

0.8( 

0.0) 

1%] 

256. 4( 

0.7) 

1.0( 

0.3) 

10. 2 ( 

0.9) 

43. 2( 

1.2) 

56. 8( 

1.2) 

New  York 

1992 

2158 

6.5( 

0.2) 

3%] 

266. 4( 

2.1)> 

3.2( 

0.5) 

20. 0( 

1.3)> 

57. 5( 

2.2)> 

42. 5( 

2.2)< 

1990 

2302 

7.7( 

0.2) 

2%] 

260. 8( 

1.4) 

2.7( 

0.4) 

15. 3(* 

0.9) 

49. 9( 

1.7) 

50. K 

1.7) 

North  Carolina 

1992 

2769 

3.2( 

0.1) 

3%] 

258. 4( 

1.2)> 

1.2( 

0.3) 

12.  K 

1.0)> 

47. 0( 

1.4)> 

53. 0( 

1.4)< 

1990 

2843 

3.6( 

0.1) 

2\] 

250.3 ( 

1.1) 

0.6( 

0.3) 

8.7  ( 

0.7) 

37. 9( 

1.4) 

62. K 

1.4) 

North  Dakota 

1992 

2314 

0.3( 

0.0) 

3k] 

283. 2( 

1.1) 

3.K 

0.5) 

29. 5( 

1.6) 

77. 9( 

1.4) 

22. K 

1.4) 

1990 

2485 

0.4( 

0.0) 

4k] 

281. K 

1.2) 

3.6( 

0.6) 

27  .3  ( 

1.8) 

75. 4( 

1.6) 

24. 6( 

1.6) 

Ohio 

1992 

2535 

5.4( 

0.2) 

4k] 

268. K 

1.5)> 

1.9( 

0.4) 

18. 1( 

1.3) 

58. 8( 

2.1) 

41. 2( 

2.1) 

1990 

2673 

5.6( 

0.1) 

2k] 

264. 0( 

1.0) 

1.7  ( 

0.3) 

14.8  ( 

1.1) 

52. 6 ( 

1.6) 

47. 4( 

1.6) 

Oklahoma 

1992 

2141 

1.5( 

0.0) 

3k] 

268. K 

1.1)> 

l.K 

0.3) 

16. 9( 

1.1) 

59. 4( 

1.6)> 

40. 6( 

1.6)< 

1990 

2222 

1.8( 

0.0) 

2k] 

263. 2( 

1.3) 

1.3( 

0.4) 

13.3  ( 

1.2) 

52.0  ( 

1.8) 

48. 0( 

1.8) 

Penn  sy  1 vani  a 

1992 

2612 

4.5( 

0.1) 

3k] 

271. 4( 

1.5)> 

2.7( 

0.5) 

21. 5( 

1.5) 

62. 1( 

1.7) 

37. 9( 

1.7) 

1990 

2528 

5.4( 

0.2) 

3k] 

266. 4( 

1.6) 

2.0( 

0.4) 

17. 2 ( 

1.3) 

56. 0( 

2.0) 

44. 0( 

2.0) 

Rhode  Island 

1992 

2120 

0.4( 

0.0) 

Ok] 

265. 9( 

0.7)> 

1.4( 

0.3) 

15.7  ( 

1.1) 

56. 4( 

1.2)> 

43. 6( 

1.2)< 

1990 

2675 

0.4( 

0.0) 

2k] 

260. 0( 

0.6) 

1.6( 

0.3) 

14. 5 ( 

0.7) 

48. 8( 

1.0) 

51. 2( 

1.0) 

South  Carolina 

1992 

2625 

1.8( 

0.0) 

2k] 

260.8 ( 

1.0) 

1.7( 

0.5) 

14.9  ( 

1.0) 

47. 8 ( 

1.3) 

52. 2( 

1.3) 

1990 

0 

0.0( 

0.0) 

Ok] 

**♦**( 

0.0) 

0.0) 

***♦*( 

0.0) 

0.0) 

0.0) 

Tennessee 

1992 

2485 

2.3  ( 

0.1) 

3k] 

258. 8( 

1.4) 

1.0( 

0.4) 

11. 6( 

1.0) 

46. 8( 

1.9) 

53. 2( 

1.9) 

1990 

0 

0.0( 

0.0) 

Ok] 

****♦( 

0.0) 

0.0) 

*****( 

0.0) 

****♦( 

0.0) 

0.0) 

Texas 

1992 

2614 

8.8( 

0.3) 

3k] 

264.6 ( 

1.3)> 

3.2( 

0.6) 

18.  K 

1.2)> 

52.7  ( 

1.5)> 

47. 3( 

1.5)< 

1990 

2542 

10. 2( 

0.3) 

3k] 

258. 2( 

1.4) 

1.6( 

0.3) 

12.7  ( 

1.1) 

45.5  ( 

1.6) 

54. 5( 

1.6) 

Utah 

1992 

2726 

1.2( 

0.0) 

3k] 

274.3 ( 

0.7) 

2.3( 

0.4) 

22. 3( 

1.0) 

66. 8( 

1.2) 

33. 2( 

1.2) 

1990 

0 

0.0( 

0.0) 

Ok] 

****♦( 

0.0) 

0.0) 

****♦( 

0.0) 

****♦( 

0.0) 

****•( 

0.0) 

Virginia 

1992 

2710 

2.8( 

0.1) 

4k] 

267. 9 ( 

1.2) 

2.7( 

0.6) 

19. 2( 

1.1) 

56.7  ( 

1.7) 

43. 3( 

1.7) 

1990 

2661 

3.0( 

0.1) 

3k] 

264.3 ( 

1.5) 

3.7( 

0.8) 

17.3  ( 

1.6) 

51. 6( 

1.7) 

48. 4( 

1.7) 

West  Virginia 

1992 

2690 

0.9( 

0.0) 

3k] 

259.  K 

1.0)> 

0.6( 

0.2) 

9.8( 

0.8) 

46. 8( 

1.6)> 

53. 2( 

1.6)< 

1990 

2600 

l.K 

0.0) 

3k] 

255. 9( 

1.0) 

0.9( 

0.2) 

9.3( 

0.8) 

41. 8( 

1.1) 

58. 2( 

1.1) 

Wisconsin 

1992 

2814 

2.3( 

0.1) 

6k] 

277. 9 ( 

1.5) 

3.2( 

0.6) 

27. 0( 

1.4) 

70. 8( 

2.1) 

29. 2( 

2.1) 

1990 

2750 

2.4( 

0.1) 

3k] 

274.5 ( 

1.3) 

3.K 

0.4) 

23. 2( 

1.4) 

65.7  ( 

1.6) 

34. 3( 

1.6) 

Wyoming 

1992 

2444 

0.3( 

0.0) 

2k] 

275. K 

0.9)> 

1.9( 

0.4) 

21. 0( 

1.1) 

67. 2( 

1.3) 

32. 8( 

1.3) 

1990 

2701 

0.3( 

0.0) 

Ik] 

272.  K 

0.7) 

1.7  ( 

0.2) 

18. 5( 

0.9) 

63.7  ( 

1.3) 

36. 3( 

1.3) 

Guam 

1992 

1496 

O.K 

0.0) 

Ik] 

235. K 

1.0)> 

0.5( 

0.1) 

5.6( 

0.6)> 

25. 3( 

1.4) 

74. 7{ 

1.4) 

1990 

1617 

O.K 

0.0) 

Ok] 

231. 8( 

0.7) 

0.4( 

0.2) 

3.8( 

0.4) 

21.7  ( 

1.0) 

78. 3( 

1.0) 

Virgin  Islands 

1992 

1479 

O.K 

0.0) 

Ok] 

222. 8 ( 

1.1)> 

0.0( 

0.1) 

0.6( 

0.3) 

9.4( 

0.9) 

90. 6( 

0.9) 

1990 

1326 

O.K 

0.0) 

Ok] 

218.7 ( 

0.9) 

O.K 

0.1) 

0.7  ( 

0.3) 

7.6( 

1.0) 

92. 4( 

1.0) 

INDICATES  A SIGNIFICANT  INCREASE  (OR  DECREASE  ■<■>  BETWEEN  1990  AND  1992 
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